WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 6 
G10H 1/00, 7/00 



Al 



(11) International Publication Number: WO 97/35299 

(43) International Publication Date: 25 September 1997 (25.09.97) 



(21) International Application Number: PCT/US97/04636 

(22) International Filing Date: 20 March 1997 (20.03.97) 



(30) Priority Data: 

08/618.906 



20 March 1996 (20.03.96) 



US 



(60) Parent Application or Grant 
(63) Related by Continuation 
US 

Filed on 



08/618,906 (CON) 
20 March 1996 (20.03.96) 



(71) Applic ant (f or all designated States except US): CALIFORNIA 

INSTITUTE OF TECHNOLOGY [US/US]; 1201 East Cal- 
ifornia Boulevard, Mail Code 315-6, Pasadena, CA 91125 
(US). 

(72) Inventors; and 

(75) Inventors/Applicants (for US only): GOODMAN, Rodney, M. 
[GB/US]; 462 E. Las Flores Drive, Aitadena, CA 91001 
(US). SPANGLER, Randall, R. [US/US]; 170 S. Chester 
Avenue #13, Pasadena, CA 91 106 (US). 



(74) Agents: HARRIS, Scott, C. et al.; Fish & Richardson P.C., 
Suite 1400, 4225 Executive Square, La Jolla, CA 92037 
(US). 



(81) Designated States: AL, AM, AT, AU, AZ, BA, BB, BG, BR, 
BY, CA, CH, CN, CU, CZ, DE, DK, EE, ES, FI, GB, GE, 
HU, IL, IS, JP, KE, KG, KP, KR, KZ, LC, LK, LR, LS, 
LT, LU, LV, MD, MG, MK, MN, MW, MX, NO, NZ, PL, 
PT, RO, RU, SD, SE, SG, SI, SK, TJ, TM, TR, TT, UA, 
UG, US, UZ, VN, ARIPO patent (GH, KE, LS, MW, SD, 
SZ, UG), Eurasian patent (AM, AZ, BY, KG, KZ, MD, RU, 
TJ, TM), European patent (AT, BE, CH, DE, DK, ES, FI 
FR, GB, GR, IE, IT, LU, MC, NL, PT, SE), OAPI patent 
(BF. BJ, CF. CG, CI, CM, GA, GN, ML, MR, NE, SN, TD 
TG). 



Published 

With international search report. 



(54) Title: MUSIC COMPOSITION 




(57) Abstract 



A music composition system (1), comprising receiving a first harmony including a first melody, analyzing the first harmony to derive 
in real-time a rule relating the tint melody to the first harmony, receiving a second melody, and applying the nile in real-time to the second 
melody to produce a second harmony relating to the second melody. 



FOR THE PURPOSES OF INFORMATION ONLY 



Codes used to identify States party to the PCT on the front pages of pamphlets publishing international applications under the PCT. 



At 


Albania 


ES 


Spain 


AM 


Armenia 


FI 


Finland 


AT 


Austria 


FR 


France 


All 


Australia 


GA 


Gabon 


AZ 


Azerbaijan 


GB 


United Kingdom 


BA 


Bosnia and Herzegovina 


GE 


Georgia 


BB 


Barbados 


GH 


Ghana 


BE 


Belgium 


GN 


Guinea 


BF 


Burkina Faao 


GR 


Greece 


BC 


Bulgaria 


HU 


Hungary 


BJ 


Benin 


IE 


Ireland 


BR 


Brazil 


IL 


Israel 


BY 


Belarus 


IS 


Iceland 


CA 


Canada 


IT 


Italy 


CF 


Central African Republic 


JP 


Japan 


CG 


Congo 


KE 


Kenya 


CH 


Switzerland 


KG 


Kyrgyzstan 


CI 


C6te d*Tvoire 


KP 


Democratic People's 


CM 


Cameroon 




Republic of Korea 


CN 


China 


KR 


Republic of Korea 


CU 


Cuba 


KZ 


Kazakstan 


cz 


Czech Republic 


LC 


Saint Lucid 


DE 


Germany 


U 


Liechtenstein 


DK 


Denmark 


LK 


Sri Lanka 


EE 


Estonia 


LR 


Liberia 



LS 

LT 

LU 

LV 

MC 

MD 

MG 

MK 

ML 
MN 
MR 
MW 
MX 
NE 
NL 
NO 
NZ 
PL 
PT 
RO 
RU 
SD 
SE 
SG 



Lesotho 

Lithuania 

Luxembourg 

Latvia 

Monaco 

Republic of Moldova 

Madagascar 

The former Yugoslav 

Republic of Macedonia 

Mali 

Mongolia 

Mauritania 

Malawi 

Mexico 

Niger 

Netherlands 
Norway - 
New Zealand 
Poland 
Portugal 
Romania 

Russian Federation 

Sudan 

Sweden 

Singapore 



SI 


Slovenia 


SK 


Slovakia 


SN 


Senegal 


sz 


Swaziland 


TD 


Chad 


TG 


Togo 


TJ 


Tajikistan 


TM 


Turkmenistan 


TR 


Turkey 


TT 


Trinidad and Tobago 


UA 


Ukraine 


UG 


Uganda 


US 


United States of America 


uz 


Uzbekistan 


VN 


Viet Nam 


YU 


Yugoslavia 


zw 


Zimbabwe 



WO 97/35299 



PCT/US97/04636 



MUSIC COMPOSITION 



Background of the Invention 

1. Field of the Invention 

The invention relates to computer-aided music 
analysis and composition. 

2. Description of the Prior Art. 

Composition and playing of music requires years of 
dedication to the cause. Many talented individuals are 
simply unable to dedicate so much of their lives to learning 
the skill. Technology has grappled with allowing non- 
practiced individuals to play music for years. Player 
pianos, automated music and rhythm organs, and electronics 
keyboards have minimized the learning curve. While these 
devices automated some parts of music reproduction to some 
extent, they severely constrained creativity. 

The player piano, for example, used a predetermined 
program indicated by holes in a roll of paper. The keys 
that were pressed based on those holes were indifferent to 
the creative ideas of an unskilled operator. 

All of these technologies force operators to rely on 
pre-packaged music originated by others. They allow very 
little creativity. Even the keynote in which the pre- 
programmed sounds are to be played is preselected. Merely 
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arranging snippets of another's music has proved a poor 
substitute for creating one's own music. 

Recently, some have tried to apply computer power in 
aid of the composer . U.S. Patent no. 5,308,915 is 
5 representative of the many systems that use a neural 

network. Computer-based music analysis and composition has 
used, for example, neural network computer technology. 
Neural networks which make use of concepts related to the 
operation of the human brain. Neural networks operate in an 
10 analog or continuously variable fashion. Some neural 

network approaches use some sort of rule-based preprocessing 
and post-processing. The knowledge which the system uses to 
make its decisions is inaccessible to the user. 

For example, takes a system with the following 

15 steps: 

Input from MIDI keyboard (10) 

i 

i 

Preprocessor puts input into a form that a neural network 
20 can understand (20) 

I 

Neural network (3 0) 

t 
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10 



Postprocessor to turn neural network output back into 

MIDI (40) 
l 

Output to MIDI sound module (50) 

The input and output that the system is sending 
may be understandable at each point in the process. 
However, ALL of the LEARNED knowledge that the system uses 
to make its decisions is hidden in the weights of the 
connections inside the neural network (30) . The inventors 
recognized that this knowledge is extremely difficult to 
extract from the network. It is difficult to phrase music in 
a form directly that can be understood by a network. All 
neural networks share the common characteristic that at some 
15 point in the process, knowledge is not stored in a 
directly-accessible declarative form. 

Another limitation commonly encountered in neural 
network approaches is related to external feedback, where 
the output of the network is used at some point in the 
20 future as input to the network. Here, the analog nature of 
the network allows it to slide away from the starting point 
and towards one of the melodies on which it was trained. 
One example is a network which learned the "blue danube". 

- 3 - 



ft 

WO 97/35299 PCT/US97/04636 



The problem with this network was that no matter what input 
you gave it, eventually it started playing the blue danube . 
The key point here is that the network may have learned the 
blue danube, but it did NOT learn HOW to write it or how to 
5 write SIMILAR but not IDENTICAL music. 

Moreover, neural networks are analog machines, and 
it is difficult to make an analog machine (a neural network) 
approximate a discrete set of data (music with a finite 
number of pitches and rhythmic positions) . 
10 One type of network used for composition is a single 

feed- forward network. This network has been used to 
associate chords with melodies. This system was described 
by Shibata in 1991. This system represents chords as their 
component tones instead of by their figured bass symbols. 
15 The network also required the entire melody at once, meaning 
it could not be performed in real-time as the melody was 
being generated by a musician. An important contribution 
from Shibata' s work is the use of psychophysical experiments 
to gauge the success of a computer compositional approach; 
20 listeners evaluated the output of the network compared to a 
table-driven harmonizing approach and indicated a measure of 
how natural the output sounded. 
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Adding recurrent connections to a neural network 
provides additional computational complexity, and allows the 
network to evolve some sense of movement through time. This 
approach has been used to teach a network a single 153 -note 
melody . 

The inventors recognized certain limitations in 
these previous studies. Neural networks have a continuous 
analog nature, which has proven to be difficult to apply to 
apply to music's a discrete set of events. Almost all music 
has some sort of regular rhythm, with notes starting either 
directly on a beat or at just a simple fraction of the beat. 
Note durations behave similarly. 

Most music is also tonal, using only a finite number 
of pitch values. Neural networks, which use a continuous or 
analog mode of operation, require excessive training to 
approximate this discrete behavior. This is a very 
inefficient use of a nueral network. 

Neural networks learn in a connective way, which is 
not conducive to determination of the rationale behind the 
learning. The inventors recognized that a music composer 
either likes or dislikes certain effects which have been 
obtained. It is an object of the present invention to allow 
the composer to interact with the computer based learning 
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system by viewing and/or modifying the results of the 
computer based learning system. It might be possible to 
modify a neural network to respond to feedback from a user 
about what that user likes or dislikes as suggested 
5 according to the present invention. Even if this were done, 
however, it would not be easy to ask the network, "I HATE 
that! Why did you do that?' 1 

Some research has been done using rule -based 
computer analyses that learn from examples. Rule-based 
10 systems are inherently discrete, easing system training. An 
example of a generic rule is shown below, with a left-hand 
side (LHS) referencing one or more attributes A* and a 
right-hand side (RHS) referencing an attribute A^ . Such a 
rule inferences the RHS attribute A^. A set of such rules 
15 is known as a rule base. 

LHS RHS 
IF A x = a lr2 and A 2 = a 2(S THEN A^ = a^^ 

U.S. Patent No. 5,418,325 describes a computer 
receiving a musical element, i.e. , a series of notes over 
20 time. This is used to build a table of rules that indicate 
which notes are most likely to follow each note received. 
Such a table is of some help to a composer of a new element 
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in order to create a series of notes that are pleasing to 
the ear. 

The inventors recognized that this will give a 
correct distribution, but will not necessarily sound good. 
Music which is done purely probabalist ically is BORING, 
i.e., it doesn't interest the ear. 

U.S. Patent No. 5,418,323 describes a system in 
which rules built from a small seed string of notes. The 
system is usually not responsive to feedback in real-time. 

The systems of U.S. Patents Nos . 5,302,777, 
5,218,153, and 4,981,544, for example, create such competing 
rules but follow through with only simplistic methods of 
making use of these rules. The present invention defines a 
new technique of weighing which allows competing rules to be 
maintained and appropriately used. 

It is hence an object of the present invention to 
provide a system which includes all of the advantageous 
aspects of the present invention - a system which operates 
using the least possible amount of computer power to learn 
musical rules and weights and apply them in real-time. The 
present invention also allows interaction with the rules, 
e.g. by viewing and/or modifying the rules that have fired. 

The system preferably stores information in the form 
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of rules, unlike the conventional learning system which 
stores information. The use of rules in addition to 
learning provides some of the benefits of both. The present 
invention uses probabilistic rules to obtain many of the 
5 capabilities of analog networks. By so doing, the present 
invention obtains all of the benefits of a rule-based 
system. This allows us to ask the system to explain its 
decisions . 

Practical operation of these systems is enhanced if 
10 the rule base is appropriately managed. Another aspect of 
the present invention defines a special real-time 
dependency pruning system which enhances the accuracy of the 
rulebase. Another aspect teaches segmenting the rulebases 
in a way which facilitates their use. Yet another aspect of 
15 the invention defines using probabilistic, e.g., not 
deterministic , rules . 

The operating techniques used by the present 
invention allow a simple algorithm with small chunks of data 
to accompany a live musician. The preferred system uses 
20 special rules which are optimized for the use according to 
the present invention. 

It is therefore an object of the invention to 
provide a music composition system useful to one lacking 
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formal training in musical arts. Another object is to 
provide a system which creates rules through analysis of 
music. Another object of the system is to provide a real- 
time composition system which applies these rules in real- 
5 time. The present system does not need to create the rules 
in real-time. In fact, the computers presently being used 
take several minutes to create the rules it later is able to 
apply to musical input with a delay of less than 1/10 
second. 

10 Another object of the invention is to provide an 

automated music composition system that creates rules 
through real-time analysis of music. In addition, it is an 
object of the invention to provide an automated music 
composition system requiring little explicitly-coded 

15 knowledge of music. It is a further object of the invention' 
to provide an automated rule-based music composition system 
in which multiple competing rules contribute to an outcome. 
Still another object of the invention is to provide an 
automated rule-based music composition system using special 

20 rules optimized to provide the best results. 

Brief Description of the Drawings 
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These and other aspects of the invention will now be 
described in detail with reference to the accompanying 
drawings , wherein : 

Fig. 1 is a diagram of hardware equipment 
5 connections according to the invention; 

Fig. 2 is an overall flowchart of a method of music 
composition according to the invention; 

Fig. 3 is a flowchart of a method of conversion to 
figured bass according to the invention; 
10 Fig. 4 shows a formula which determines a J-measure 

according to the invention; 

Figs. 5-8 depict a detailed flowchart of a method of 
rule generation according to the invention; 

Fig. 9 is a flowchart of a method of harmonization 
15 according to the invention; 

Fig. 10 is a flowchart of a method of conversion to 
MIDI according to the invention; and 

Figs. 11-14 are musical charts representing products 
of music composition according to the invention. 



Description of the Preferred Embo diments 
It should be understood that many of the techniques 
described herein are intended to be carried out in software 

- 10 - 
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on a computer-based system, such as a personal computer or 
synthesizer. The following describes the functions that are 
carried out. 

The music composition system of the present 
invention automatically learns rules representing a 
particular style of music and uses those rules to generate 
new music in the same style. The generated accompaniment 
can be for a performing musician in real-time. 

Fig. 1 shows the system using a standard 486SX 
computer 10 running a standard operating system, e.g. , DOS 
or a multithreaded operating system such as Microsoft 
Windows NT. User input in, e.g. , MIDI format can be 
accepted through the computer keyboard 3 0 or through any 
synthesizer or musical keyboard connected to the computer by 
a standard MIDI interface. The system's output is sent via 
the MIDI interface to a synthesizer 50 for playback. 

The application examples below provide a context for 
the detailed information to follow. For instance, the 
system can operate as a computerized expert trained using 
examples of a particular musical style. Students attempting 
to write music in the particular style can ask the 
computerized expert not only to check their compositions for 
errors but also to suggest alternatives. Because the system 
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is rule-based, the computerized expert based on the system 
can also provide explanations showing why the suggestions 
overcome the errors. 

The system can also allow comparison of two or more 
different composers' works by generating a rule base for 
each composer. Furthermore, a musical piece can be checked 
against a particular composer's known rule base to determine 
whether the piece was in fact authored by that composer. 

Soundtracks can be generated using the system. The 
system creates rule bases, i.e. is trained, from musical 
pieces known to provoke certain feelings or having certain 
styles. These rule bases can be used subsequently to 
generate music appropriate for particular situations. 

The system can make a small number of musicians 
sound like a large orchestra. For example, additional 
musical lines generated from an existing four- or five -part 
harmony can be fed to the synthesizer to make a string 
quartet sound like an entire string orchestra. 

Along the same lines, the system can simulate a 
rock-n-roll band, allowing an aspiring musician to play 
along. With the aspiring musician's musical instrument 
plugged into the computer and the style of each member of, 
say, The Beatles musical group encoded into an individual 

- 12 - 
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rule base, the system can accompany the aspiring musician in 
much the same way as The Beatles would have. Furthermore, 
trained on a missing member's style, the system can take the 
place of that member in a musical group's subsequent 
recordings . 

The system is capable of learning all of its musical 
knowledge from sample pieces of music. This capability 
provides flexibility, allowing application of the system to 
musical styles not originally planned. In addition, because 
the rules are determined and applied automatically, 
requiring no hand- tuning, the system works well for users 
lacking much technical knowledge of music. Finally, able to 
accept industry- standard MIDI song files as musical input, 
the system can generate, quickly and easily, series of rule 
bases representing the styles of various composers. Control 
over rule generation is available for advanced users of the 
system. 

A particularly useful feature of the system is its 
ability to demonstrate the basis of its decisions by listing 
the rules extracted during training. Such listings make the 
system useful as an interactive aid for teaching music 
theory and as a tool for historians attempting to understand 
the creative processes of composers such as Bach and Mozart. 

- 13 - 
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A further indication of the system's power is its 
ability to resolve conflicts when two or more rules call fo 
different outcomes. The system employs several such 
schemes, including rule weighing and real-time dependency 
pruning . 

The present invention provides efficient ways of 
generating and activating, or firing, rules, allowing the 
system to operate in real-time using everyday computers. 
Thus any live musician can use the system to generate 
accompaniment. The real-time aspect of the system also fit 
well with other interactive tasks, such as teaching music 
theory. 

An example of the system's work is shown below. 
Using the well-known Bach chorales as input, the system 
generates the five rules below, which are some of the most 
commonly-used rules in classical Bach harmony, typically 
appearing in any first -year music theory textbook. 

1. IF MelodyO E THEN FunctionO 1 
AND Functionl V 

0 (G Major to C Major) 

2. IF MelodyO F THEN FunctionO IV 
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AND Functionl V 
(G Major to F Major) 

3. IF Functionl V THEN InversionO II 
AND FunctionO IV 

5 4. IF Functionl V THEN InversionO 10 

AND FunctionO I 

5. IF FunctionO vii07 THEN InversionO II 

The system does not use a textbook but learns such rules on 

its own, as explained below. 
0 Fig. 2 is a flowchart showing the operation of the 

system. The flowchart shows the overall operation, 

including : 

Conversion to figured bass (step 1000) , 
Generation of example tables (step 1010) , 
5 Derivation of rules from examples (step 1020) , 

Filtering and segmentation of rules (step 1030) , 
Subsumption pruning of rules (step 1040) , 
Generation of dependence data (step 1050) , 
Harmonization using rules (step 106 0) , and 
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Conversion to MIDI (step 1070) . 



The preferred system works with musical information 
represented in a variation of a form known as figured bass. 
The figured bass form has been used frequently by composers 

5 to present a piece's harmonic information without stating 
the precise location, duration, and pitch for every single 
note. In classical form, a figured bass states the melody 
and represents the underlying harmony as a series of chords. 
Each chord is specified by its function in the key of the 

10 piece of music, written as a Roman numeral or "figure, " and 
the pitch which is being played by the bass voice. There 
are usually several ways of voicing any given figure, i .e . , 
turning the figured bass representation back into notes. The 
preferred system uses an extended form of figured bass that 

15 includes the chord notes played by all the voices, which 

allows the system to turn the figured bass back into notes 
while playing. 



- Conversion to figured bass 

The conversion step 1000 converts music represented 
2 0 in MIDI file format into the figured bass format needed by 
the steps that follow. The MIDI file format is a 
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specification for storage and transmission of musical data. 
Under MIDI, musical data is arranged as a stream of events 
occurring at specified intervals. The following is a 
typical stream of MIDI data: 



Header format=0 ntrks=l division=240 
Track start 

Delta time=0 Time signature=3/4 MIDI -clocks /click=2 4 32nd 

notes/2 4 -MIDI -clocks=8 
Delta time=0 Tempo, microseconds-per-MIDI-quarter-note=41248 
Delta time=0 Meta Text, type=0x01 (Text Event) leng=23 

Text = <Chorale #001 in G Major> 
Delta time=480 Note on, chan=l pitch=67 vol=88 
Delta time=0 Note on, chan=2 pitch=62 vol=72 
Delta time=0 Note on, chan=3 pitch=59 vol=88 
Delta time=240 Note off, chan=4 pitch=43 vol=64 
Delta time=0 Note off, chan=3 pitch=59 vol=64 
Delta time=0 Note off, chan=2 pitch=62 vol=64 
Delta time=0 Note off, chan=l pitch 67 vol=64 
Delta time=0 Note on, chan=l pitch=67 vol=81 
Delta time=0 Note on, chan=2 pitch=62 vol=75 
Delta time=0 Note on, chan=3 pitch=59 vol=88 
Delta time = 0 Note on, chan==4 pitch=55 vol = 60 
Delta time=240 Note off, chan=4 pitch=55 vol=64 
Delta time^O Note off, chan=3 pitch=59 vol=64 
Delta time=0 Note off, chan=2 pitch=62 vol=64 
Delta time=0 Note on, chan=2 pitch=64 vol=58 
Delta time=0 Note on, chan=3 pitch=60 vol=78 
Delta time=1920 Meta Text, type=0x01 (Text Event) leng=7 
Text = <Fermata> 



Each line in the stream is an event. For example, 
in the line "Delta time=240 Note off, chan=4 pitch=43 
vol=64," the phrase "Delta time=240" means that the line 
starts executing 240 MIDI -clocks of time after the last li 
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started executing. "Note off" indicates that the note 
presently being played by channel, i.e. , voice "4" is to be 
turned off. 

The significant events in the sample data are listed 
in the following table. 



Event 
Time 

signature 



Note 

on/Note 

off 



Function 



Gives 

information 
about the 
timing of 
the piece 



Met a Text 



Turns a note 
on or off 
for a 
specific 
voice 



Allows 
arbitrary 
messages to 
be sent 



Relevant 
Parameters 

Time 

signature 



32nd- 
notes/24- 
MIDI- 
clocks 

Channel 



Pitch 



Text 



Meaning 

Needed to convert 
beats into measures 
and to determine beat 
accents . 

Needed to convert 
current t ime into 
beat number. 

Which voice is 
changing ( l=soprano , 
2=alto, 3=tenor, 
4=bass) . 



Which note is 
changing (pitch=60 is 
middle C 3 ) . 

"Chorale #001 in G 
Major" gives the name 
and key of the piece. 
"Fermata" states that 
there is a fermata on 
the chord starting at 
that time. 
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The inventors prefer using musical data that is not 
in the MIDI format as their input for musical analysis. In 
MIDI data, which notes are being played at a given point in 
time is difficult to determine because the durations of the 
notes are not explicitly coded . Rhythmic structure is 
difficult to determine as well. The MIDI format is 
sensitive to the exact notes being played. For example, 
transposing the piece, i.e. , adding a fixed pitch interval 
to all notes, changes every pitch in the music's MIDI data 
stream. If a piece is transposed up a semitone (from C to 
C- sharp, for example) , every single pitch in the MIDI data 
changes. Even minor changes in the voicing of a chord have 
radically different representations in the MIDI data. For 
example, a C Major chord (C, E, G, C) could have pitches 
{60, 64, 79, 84}, or {67, 72, 76, 84}. The two voicings 
sound almost identical and have similar functions, but share 
only one common pitch. This problem is solved by 
transforming the data into a figured bass format. 

The figured bass format used by the system more 
concisely states the harmonic content and rhythmic 
information for an accompaniment. In figured bass format as 
opposed to MIDI format, music is organized in terms of 
chords and beats instead of individual transition events. A 
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typical figured bass corresponding to the first few chords 
of MIDI data listed above, follows. 





MEL 


FUNC 


IN 


TP 


AP 


SP 


DUK 




-J 




I 


10 


Tl 


A2 


SO 


2 


un 




C 


I 


10 


TO 


Al 


SO 


2 


acc 




c 


IV 


11 


TO 


Al 


S2 


1 


un 




c 


vi 


10 


T2 


AO 


si 


1 


n 


10 


G 


V 


11 


T2 


AO 


SO 


z 


un 




E 


I 


10 


TO 


A2 


SI 


2 


ACC 




E 


iii 


11 


T2 


Al 


so 


1 


un 




D 


V 


10 


Tl 


AO 


S2 


1 


n 


15 


C 


vi 


10 


Tl 


A2 


SI 


2 


un 




c 


IV 


10 


TO 


Al 


S2 


1 


ACC 




c 












1 


n 




c 




13 


TO 


Al 


S2 


1 


un 


20 


D 


vii07 


11 


T2 


AO 


SI 


1 


n 


E 


I 


10 


T2 


AO 


SI 


2 


un 




D 


V 


10 


TO 


Al 


S2 


4 


FERM 






The first 


column, 


with 


the 


heading 


MEL , lists 



25 pitch played by the soprano, which is the melody note of tl 
piece. Next is the column headed FUNC, which is the chord 
function or figure. The most common functions in a major 
key in the work of Bach, for example, are listed in the 
following function table, which is only a subset of the 

30 total list of functions used by the system. 
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Function Chord Name Pitches 

I C Major C, E, G 

17 C7 C, E, G, B-flat 

ii D minor D, F, A 

5 v / v D Major D, F- sharp, A 

iii E minor E, G, B 

v / vi E Major E, G-sharp, B 

IV F Major F, A, C 

V G Major g, B, D 

10 v ? G/ B, D, F-sharp 

vi A minor A, C, E 

vii07 B diminished 7th B, D, F, A-flat 



The middle set of four columns, headed IN, TP, AP, 
and SP, indicate the positions, respectively, of the bass 

15 voice, or inversion; the tenor voice; the alto voice; and 
the soprano voice. The positions are numbered from 0 to 3, 
wherein 0 indicates the first pitch listed in the function 
table above and 3 indicates the fourth pitch. For example, 
again using the function table above, in the key of C major, 

20 a V7 chord with positions 10 Tl A3 SO would contain, in 
order, the pitches G, B, F-sharp, and G. Use of this 
position notation provides the system with musical data 
that, while allowing easy reconstruction of the original 
pitches, is key- independent , because if a piece of music is 

25 transposed, its voice positions remain unchanged. 
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In addition, since figured bass reduces the number 
of possibilities from twelve pitches to four positions, the 
overall complexity of the set of musical data is reduced. 

The next column, under the heading DUR, shows the 
duration of the particular chord. Lastly, the column headed 
ACC also indicates a timebase, by displaying the accent to 
be placed upon the chord. Under the ACC column, the 
following notations have the following meanings: "FERM", 
standing for fermata or held chord, indicates the strongest 
accent; 11 ACC" signals that the chord begins at the start of 
an accented beat; " un " specifies that the chord begins on an 
unaccented beat; and "n" means that the chord does not begin 
at the start of a beat. 

Fig. 3 shows converting a musical piece described in 
a MIDI file to the desired figured bass form. The system 
scans through the MIDI file and assembles all of the pieces 
together to determine which notes are being played by the 
voices, viz , bass, tenor, alto, soprano, and at which times 
(step 1000a) . The system then extracts the key of the piece 
from the initial MIDI text event, an example of which is 
shown in the sample MIDI stream above (step 1000b) . 
Standardizing to simplify later analysis and to ease 
comparisons of different pieces, the system transposes the 
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piece to the key of C Major, with all of the pitches 
changing appropriately (step 1000c) . Next, beginning a new 
chord whenever a voice changes pitch, the system segments 
the piece into chords (step lOOOd) . 

Segmented into chords, the piece appears as follows. 



TIME 


DUR 


B 


T 


A 


S 


000 
004 












2 { C3 


E4 


G4 


C5 


006 
006 












2 


C4 


E4 


G4 


C5 


008 


1 


A3 


F4 


A4 


C5 


009 


1 


A3 


E4 


A4 


C5 


010 


2 


B3 


D4 


G4 


G5 



Representing one timestep, i.e. . one-eighth of a 
note, and one chord, each line contains information about 
when the chord was started, its duration, and which note is 
being played in each voice. Next, determining the melody 
pitch by taking the soprano note without the octave, the 
system also determines the accent of each chord (step 
lOOOe) . The accent is based on the time a chord starts and 
the time signature of the piece. For example, in 3:4 time, 
the time signature for the sample listed above, a measure is 
6 timesteps long because each timestep is one-eighth of a 
note. Thus, accented beats occur every 6 timesteps and 
unaccented beats occur every 2 timesteps, as indicated in 
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the table below, wherein n is an integer representing the 



10 



15 



20 



25 



measure number . 

Time 

6n+0 
6x1+1 
6n+2 
6n+3 
6ii+4 
6n+5 



Accent 

ACC 
n 

un 
n 

un 
n 



TIME 

000 

004 

006 

006 

008 

009 

010 



DUR 
2~""{ 



B 

C3 



T 
E4 



A 
G4 



S 

C5 



MEL ACC 
C un 



2 
1 
1 
2 



C4 
A3 
A3 
B3 



E4 
F4 
E4 
D4 



G4 
A4 
A4 
G4 



C5 
C5 
C5 
G5 



C 
C 
C 
G 



ACC 

un 

n 

un 



Next, the system identifies a timestep with a 
particular known chord by attempting to match the 
information at each timestep with a known chord, i.e.., 
matching if all pitches being played could be part of that 
known chord (step lOOOf ) . For example, using the table 
above and a list of 120 common chords sufficient to identify 
99% of all chords occurring in Bach's music, the chord at 
timestep=8 is identified as an F Major chord because all of 
its pitches are either F, A, or C. A chord unable to be 
identified as a known chord is marked as such, because such 
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10 



15 



a chord is usually the product of a passing tone or other 
ornament and has no significant function in the piece. 
Updated, the table then appears as follows. 

DUR B T A S MEL ACC RT TYPE 
2 { C3 E4 G4 C5 } C un C Major 



TIME 
000 
004 
006 
006 
008 
009 
010 



2 


C4 


E4 


G4 


C5 


• C 


1 


A3 


F4 


A4 


C5 


■ C 


1 - 


A3 


E4 


A4 


C5 


c 


2 


B3 


D4 


G4 


G5 ] 


G 



ACC C 

un F 

n A 

un G 



Major 
Major 
Minor 
Major 



Next, the system determines the position of each 
voice by comparing the pitch of each voice with the pitches 
allowed in the identified known chord (step lOOOg) . Thus, 
in the current example, the chord at timestep=8 has pitches 
{A, F, A, C}, which correspond to positions {il, TO, Al, 
A2}, resulting in the following determinations of voice 
positions . 



TIME 
20000 
004 
006 
006 
008 
25009 
010 



DUR B T 



A S MEL ACC RT TYPE IN TP AP 



SP 



2 { C3 


E4 


G4 


C5 } C 


un 


C 


Major 


10 


Tl 


A2 


SO 


2 


C4 


E4 


G4 


C5 


C 


ACC 


C 


Major 


10 


Tl 


A2 


SO 


1 


A3 


F4 


A4 


C5 


C 


un 


F 


Major 


11 


TO 


Al 


S2 


1 


A3 


E4 


A4 


C5 


C 


n 


A 


Minor 


10 


T2 


AO 


SI 


2 


B3 


D4 


G4 


G5 


G 


un 


G 


Major 


11 


T2 


AO 


SO 



Now the system identifies a function associated with 
each chord, by comparing the root and type of each chord 
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with a table of common functions such as the Bach-related 
one described above, (step lOOOh) . When a chord is unable 
to be matched with any of the common functions, its function 
is marked as unknown, indicating that the chord may be the 
5 result of an ornament serving no harmonic function. 

Finally, since not needed in the figured bass 
notation, information about absolute time and voice pitch is 
discarded, leaving the following as the output of the 





conversion from 


MIDI 


to figured 


bass 


(step 


lOOOi) . 


10 


MEL 


FUNC 


IN 


TP 


AP 


SP 


DUR 


ACC 




C 


I 


10 


Tl 


A2 


SO 


2 


un 




C 


I 


10 


Tl 


A2 


SO 


2 


ACC 


15 


c 


IV 


11 


TO 


Al 


S2 


1 


un 




c 


vi 


10 


T2 


AO 


SI 


1 


n 




G 


V 


11 


T2 


AO 


SO 


2 


un 



In addition to the chord-based conversion just 
2 0 described, the system can use beat -based conversion. Beat- 
based conversion takes advantage of harmonic functions 
usually changing only minimally between beats, not within a 
single beat. Ornaments usually relate to only half of a 
beat and the chords formed from them are less correlated 
25 with the surrounding music than the chords relating to the 
other half of the beat. The examples which include 
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information from ornament chords tend not to correlate well 

with other examples and thus produce only weak rules. 

The beat-based conversion method is more complex 

than the chord-based method because beat -based conversion 

5 examines each chord which is part of a beat and generates an 

example assuming that the chord was the significant chord 

for that beat. All examples for a timestep then have their 

weights normalized so that the total weight for each 

timestep is one. The segment of figured bass listed above 

10 would produce the following examples. 

%NAME 0 Weight 

%NAME 1 Functionl 

%NAME 2 FunctionO 

1.0 - I 

15 1.0 I I 

0.5 I IV 

0.5 I vi 

0.5 IV V 

0.5 vi V 

20 This is fairly straightforward when the examples are using 
only one previous beat of data. However, if an example set 
is. built from the current beat and four previous beats, and 
each beat has two chords, i.e. , an ornament chord and the 
real chord, then each beat results in a quantity of samples 

25' equal to 2 raised to the fifth power, i.e. . 32 examples, 

each with weight 0.03125. Therefore, excepting example sets 
with only a small time window, a beat -based example set uses 



WO 97/35299 



PCT/US97/04636 



a great deal more memory than a standard chord-based example 
set . 



- Generation of example tables 

Rules are generated based on examples that are 

created from the figured bass data. Each example includes 

the data necessary to agree or disagree with a potential 

rule, including information about previous timesteps. 

Examples in the table can also be weighted, so that they can 

count for more or less than a normal example. As indicated 

below in the following" illustrative table, some examples 

have double the weight of other examples. Each example 

includes information about the melody and chord function 

used at the current times tep and at the previous two 

timesteps . 

%NAME 0 WEIGHT 

%NAME 1 DurationO 

%NAME 2 Melody 2 

%NAME 3 Melodyl 

%NAME 4 MelodyO 

%NAME 5 Function2 

%NAME 6 Functionl 

%NAME 7 FunctionO 

1.0 1 c C C I i iv 

1.0 1 C C C I I vi 

1.0 2 C C G I IV V 
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1 . 


o 


2 


c 


c 


G 


I 


vi 


y 


1 . 


0 


2 


c 


G 


E 


IV 


v 


I 


1. 


0 


2 


c 


G 


E 


vi 


V 


I 


1. 


0 


1 


G 


E 


E 


V 


I 


iii 


1. 


0 


1 


G 


E 


D 


V 


I 


V 


1. 


0 


2 


E 


E 


C 


I 


iii 


vi 


1. 


0 


2 


£ 


D 


C 


I 


V 


vi 


0. 


5 


1 


E 


C 


C 


iii 


vi 


IV 


0. 


5 


1 


D 


C 


C 


V 


vi 


IV 


0 . 


5 


1 


C 


c 


D 


vi 


IV 


vii07 


0. 


5 


2 


c 


D 


E 


IV 


vii07 


I 


1 . 


0 


4 


D 


E 


D 


vii07 


I 


V 



To generate examples from a figured bass, the system 
moves a window down the list of chords, copying only certain 

15 pieces of information at each timestep. For instance, 

working with the sample figured bass conversion output data 
above to generate an example table using fields FunctionO 
and Functionl, i.e. , the chord functions at the current and 
previous timestep, respectively, the system would produce 

20 the following. Each line is an example containing the 
attributes Functionl and FunctionO . 



Functionl 


FunctionO 




I 


I 


I 


I 


IV 


IV 


vi 


vi 


V 



- Derivation of rules from example tables 
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While generating rules from examples, the system 
uses a J-measure defined as shown in Fig. 4. 

The J-measure represents a balance of the amount of 
information a rule contains and the probability that the 
5 rule will be able to be used. Since a rule is less valuable 
if it contains little information, the J-measure is low when 
the rule's probability of being correct is low, i.e., when 
p(x|y) is about the same as p(x). A rule which fires only 
extremely rarely is of minimal use even if is extremely 
10 conclusive. For instance, a rule base containing many 

always -correct rules, each useful on only one example, tends 
to perform extremely well on a training set but dismally in 
general . 

An important part of the present invention is the 
15 generation technique that is used herein. The technique 

includes sorting the examples before extracting the rules 
therefrom. This has greatly improved the speed of the 
technique, as described herein. 

Rules are generated using preset parameters which 
20 can be modified by the user if necessary. To prevent 

generation of rules based on too few examples, the system 
uses a parameter N min which denotes the minimum number of 
examples with which a rule should agree. 
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A list of examples E lt E 2 , ... E NEX is used to 
generate the rules. The value of attribute i for example E. 
is denoted e J#i . 

Each rule generated preferably has a minimum J- 
measure J min and fires correctly a minimum fraction of the 
time p min . 

On the output or right-hand side of the rule, the 
rule that is generated inferences an attribute A RHS taking 
integer values a RHS(1 , a^, ... a RHS#NRHSV# where NRHSV stands 
for the number of possible RHS values. Similarly, the 
attributes allowed on the input or left-hand side of the 
rule, A lt A 2 , ... Anlhs, take on \A i \ integer values a ifl , a it2 

• - « a i,NLHSV- 

The complexity of the system is reduced using a 
maximum rule order 0^, representing the maximum number of 
attributes allowed on the left-hand side. 

The system uses an array NR of size NRHSV, as 
described herein. 

The processing according to the present invention 
uses substeps (Figs. 5-8) for each possible combination of 
LHS attributes (steps 1020a-b) . The system adds a hash 
column H to the table, each element h, of which is 
preferably a signed 32 -bit integer corresponding to an 
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example (step 1020c) . Of course, more detailed 
calculations would require more bits. Using a combination 
of LHS attributes A,, A 2 , A 5/ for instance, h ± is determined 
as follows (steps 1020d-h) . 

^ = e i>s + iA 5 |(e i(2 + |A 2 |(e ifl )) (step 1020g) 

When an attribute is unknown, h; is set to -1 (step 1020h) . 

Next, the system adds a column X of indices to the 
table: Xi = i (step 1020i) . The table is quicksorted to 
group the lines of the table by hash value (step 1020j) . 
Column X is actually what is sorted, because each entry in 
column X is only a two-byte integer. The index is only a 2- 
byte integer if fewer than 65535 examples are being 
classified. Otherwise, a 4-byte integer is preferably used. 
This saves on the amount of memory moved during the sort, 
which in turn saves time . 

After sorting, the system then searches down the 
table to generate a preliminary rule for each hash value 
(steps 1020k-l) . The elements of array NR, denoting all 
possible RHS values a^, are used to indicate correspondence 
between RHS values a^s and hash values h. Array element 
NRLaR^j] is incremented when the hash value hj for the 
current line is the same as the hash value h for the 
previous line (steps 1020m-n) . If the two hash values are 
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different, the system notes a preliminary rule relating to 
the previous hash value and then sets all element arrays NR 
to zero except for NRta^^] which is set to one. 

The preliminary rules linking each hash value to one 
or more a^ are subjected to a series of tests using the 
parameters mentioned above (steps 1020o-s) . A preliminary 
rule is rejected if the number of examples corresponding to 
the hash value is less than N min (step 1020r) or if the 
particular a^ did not occur in more than p min of the 
examples corresponding to the hash value (step 1020q) . 
Finally, the system retains the rule only if its J-measure 
is above a J-threshold (step 1020s) . 

Rules are stored in a rule array (step 1020t) . The 
rule array has a certain size, so it can only hold a 
predetermined number of rules. If the rule array overflows 
when a new rule is added (step 1020u) , the system drops the 
rule with the lowest J-measure, which becomes the new J- 
threshold (step 1020v) . After all examples in the table 
have been considered, the result is a rule base for the 
selected attribute. 

The following is a simplified illustration further 
explaining the derivation of rules and using the example 
table and parameters listed below. 
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Attn 


Attr2 


Attr3 


A 


A 


B 


A 


B 


C 


C 


B 


C 


c 


A 


B 


A 


B 


C 


B 


B 


C 


C 


C 


A 


B 


A 


C 



In this illustration, N min is set to 2, which means 
that a rule which correctly predicts only one example is 
discarded. The attribute values are found by reading across 
each example, e.g. . e 2tl =A, e 2 . 2 =B, e 3 . 3 =C. The minimum J- 
measure is 0.001 and the minimum fraction of the time a rule 
should be correct is p min -0.50, i.e. , a rule should be right 
half the time. 

In this case, Attr3 is to be predicted using Attrl 
and Attr2. In other words, is Attr3 , taking on values 

aRHs,i= A ' a*Hs, 2 =B, agHs.j-C, because, in this example, Attrl and 
Attr2 also have the same possible values A,B,C. Since there 
are 3 possible values for each attribute, | Attrl J = |Attr2| 
= |Attr3l = 3. When dealing with the attribute values as 
numbers, the following are used: A=0, B=l, C=2 . The maximum 
rule order 0^ being 2, rules can appear in either of the 
following two forms. 
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(1st order rule) if (terml) then (term2) 

(2nd order rule) If (terml) and (term2) then (term3) 

First, the system produces hash values for the 
first -order rules which are of the following form. 

If Attrl= (something) then Attr3= (something) 

The first column in the table is an index identifying the 
particular example line. 



1. 


A 


A 


B 


hash=0 


2 . 


A 


B 


C 


hash=0 


3 . 


C 


B 


C 


hash=2 


4 . 


C 


A 


B 


hash=2 


5. 


A 


B 


C 


hash=0 


6. 


B 


B 


C 


hash=l 


7 . 


C 


C 


A 


hash=2 


8 . 


B 


A 


C 


hash=l 



Sorting the examples based on hash value produces 
the following list. 
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1. 


A 


A 


B 


hash=0 


2. 


A 


B 


C 


hash=0 


b . 


A 


D 




hash=0 


6 . 


B 


B 


C 


hash=l 


8 . 


B 


A 


c 


hash=l 


3 . 


C 


B 


c 


hash=2 


4 . 


C 


A 


B 


hash=2 


7. 


C 


C 


A 


hash=2 



The system will try to make a rule for the examples 
10 with hash=0. This will provide the following possible 
rules . 

If Attrl=A then Attr3=B (correct 33% of the time) 
If Attrl=A then Attr3=C (correct 67% of the time) 

The first of the two rules is discarded because 33% , 
15 or 0.33 as a fraction, is less than 0.50, the minimum 
probability p min allowed for a rule to be retained. 
Proceeding similarly for the hash values 1 and 2 provides 
the following retainable rules. 

If Attrl=A then Attr3=C (correct 67% of the time) 
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If Attrl=B then Attr3=C (correct 100% of the time) 

Next, generating the hash value based on Attr2 instead of 
Attrl produces the following list. 



1 . 


A 


A 


B 


hash=0 


4 . 


C 


A 


B 


hash=0 


8 . 


B 


A 


C 


hash=0 


2 . 


A 


B 


C 


hash=l 


3 . 


C 


B 


C 


hash=l 


5 . 


A 


B 


C 


hash=l 


6. 


B 


B 


c 


hash=l 


7 . 


C 


C 


A 


hash=2 



The following rules would be retained. 

If Attr2=A then Attr3=B (correct 67% of the time) 
If Attr2=B then Attr3=C (correct 100% of the time) 

15 On the other hand, the following rule is correct 

sufficiently often but still needs to be discarded because 
it has only one supporting example, #7, and thus fails to 
satisfy the N min threshold. 
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If Attr2=C then Attr3=A (correct 100% of the time) 



The retained rule list now appears as follows. 



If Attrl=A then Attr3=C 
If Attrl=B then Attr3=C 
If Attr2=A then Attr3=B 
If Attr2=B then Attr3=C 



(correct 67% of the time) 

(correct 100% of the time) 

(correct 67% of the time) 

(correct 100% of the time) 



Next are the rules which use both Attn and Attr2 . 
In this case, since Attr2 has 3 possible values, the hash 
value for an example is calculated by the following 
equation, producing the table below. 

hash = 3+(Attrl's value) + (Attr2's value) 



1. A A B hash= 3*0+0 = 0 

2. A B C hash= 3*0+1 = 1 

5. A B C hash= 3*0+1 = 1 
8. B A C hash= 3*1+0 = 3 

6. B B C hash= 3*1 + 1 = 4 
4. C A B hash= 3*2+0 = 6 

3. C B C hash= 3*2+1 « 7 
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7- C C A hash= 3*2+2 = 8 



The only rule that is retained from this hash array using 
the criteria is the following, because no other hash value 
corresponds to a sufficient number of examples. 

5 If Attrl=A and Attr2=B then Attr3=C (correct 100% of the 
time) 

The resulting fully updated rule base appears as 

follows . 



If Attrl=A then Attr3=C 
10 If Attrl=B then Attr3=C 
If Attr2=A then Attr3=B 
If Attr2=B then Attr3=C 
If Attrl=A and Attr2=B 1 
time) 



(correct 67% of the time) 
(correct 100% of the time) 
(correct 67% of the time) 
(correct 100% of the time) 
an Attr3=C (correct 100% of the 



This procedure result in a rulebase. Computationally, 
this algorithm is very appealing because of its simplicity. 
Each set of LHS values is considered only once. At the time 
of consideration, all examples with that LHS are 
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consecutive, so it is not necessary to search through the 
entire example set to determine the number of examples with 
which a potential rule agrees. Memory consumption is also 
reasonable, scaling linearly with the number of examples. 

- Filtering and segmentation of rules 
The rule bases are preferably filtered and/ or 
segmented to form multiple more efficient rule bases. When 
it is known that a certain attribute is crucial to 
determining the RHS value for the rule base, filtering is 
used to force all rules contained therein to use that 
attribute. For example, the system has been used to filter 
out rules which disregard the current melody note in 
determining the current chord function. 

Segmentation is done when filtering a rulebase would 
reduce the domain which the rulebase covers. As in 
filtering, rules are grouped based on the presence or 
absence of an attribute on their 'LHS. However, the rules 
lacking the desired attribute are placed in a second 
rulebase, rather than being removed. When a series of 
segmented rulebases is used to inference a result, the 
rulebase with the desired attribute is tried first. If no 
rules in that rulebase can fire, the rulebase lacking the 
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desired attribute is tested. This gives the benefits of 
filtering since rules with the desired attribute are not 
overwhelmed by rules lacking the attribute. However, unlike 
filtering, this technique does not involve a loss of domain 
size, since the less desirable rules are not deleted, just 
prevented from firing unless there is no alternative) 

- Subsumption pruning of rules 

After being filtered or segmented, a rule base might 
still contain many rules that contribute nothing, or 
contribute so little that they are not worth keeping. 
Subsumption pruning removes such unneeded rules using the 
technique described herein. 

At step 500, rules are reviewed to determine whether 
two rules A and B predict the same RHS attribute and value. 
If so, rule B is removed from the rule base if 

(1) the left-hand side of rule B has more attributes 
than the left-hand side of rule A, 

(2) every attribute on the left-hand side of rule A 
is present and has the same value on the left-hand 
side of rule B, and 

(3) rule A is correct at least as often as rule B. 
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Since rule B adds no new information in this case, the 
system becomes more efficient by removing such a rule. 

Subsumption pruning should be done after any filtering 
and segmentation. If rule A in the previous example were 
5 filtered out, then, in retrospect, rule B should not have 
been removed: we have lost information. 

- Generation of dependence data 

For the rule-based system to work properly, all 
rules which are allowed to fire should be independent of 

10 each other. Otherwise, one good rule could be overwhelmed 
by the combined weight of twenty mediocre but virtually 
identical rules. To prevent this problem, each rule base is 
analyzed to determine which rules are dependent with other 
rules in the same rule base. Two rules are considered 

15 dependent if both rules fire in more than half of the 
examples that cause at least one of them to fire. 

To allow real-time independence pruning, the system 
maintains for each rule a list of dependent rules with lower 
J-measures. Independence pruning should be done in real- 

2 0 time, because removing all dependent rules at the time of 

rule base creation degrades its quality. For instance, if a 
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rule base contains only the following two rules which are 
dependent and the value for A : is currently unknown, the 
system cannot inference a value for A at all without the 
second rule. 

5 IF A x = a 12 THEN A RHS = a^^ with J-measure 0.013 

IF A 2 = a 2fS THEN A RKS = a ms ^ with J-measure 0.009 
Given a group of dependent rules, real-time 
independence pruning prevents the firing of all but the rule 
with the highest J-measure. The system uses an array F with 
10 all values f initially set to zero, indicating at first that 
all rules are allowed to fire. When a rule R L fires while 
the system is checking rules in order of decreasing J- 
measure, the system adds the weight of rule R A to the 
overall weight of the RHS value and then sets to non-zero 
15 the values f 3 for all rules Rj dependent with rule R t . 

More specifically, the operation proceeds as 
follows . 

1. Consider two rules RA and RB which predict the same RHS 
and value . 

20 2. Let A be the set of examples for which rule RA fires. 
3. Let B be the set of examples for which rule RB fires. 
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4. Define the overlap OAB as the number of examples for 
which both RA and RB fire; divided by the number of 
examples for which either RA or RB fires. 

5. If OAB > 0.5, the rules are dependent. 

Each rule is associated with a list of lower 
J-measure rules which are dependent with the rule. This 
list is used in real time independence pruning as described 
herein . 

It would seem at first that it would be easiest to 
remove all dependent rules at the time a rulebase is 
created. However, this actually degrades the quality of the 
rulebase. As an example, assume a rulebase containing only 
the following two rules, and assume the rules are dependent: 
IF Al =» al , 2 THEN ARHS = aRHS,3 with J-measure 

0. 013 

IF A2 = a2,5 THEN ARHS = aRHS,3 with J-measure 

0 . 009 

Now assume we are trying to inference ARHS and that the 
value of Al is currently unknown. Only the second rule 
would be able to fire. However, if we removed the second 
rule at the time of rulebase creation, no rules would be 
able to fire and we would not be able to inference a value 
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for A. We can avoid this problem by only independence 
pruning those rules which can fire for a given LHS. 
- Rulebase interaction 

An important part of musical composition is the 
ability to reinforce good sounds, and prevent bad sounds. 
Interaction buttons 60 facilitate this operation. The 
interaction buttons allow the contents of the rulebase to be 
modified based on whether the user likes or does not like a 
certain thing that the computer has done. 

For example, if the computer makes a chord which is 
not pleasing the user's ear, it indicates that the rules 
governing that chord are not desirable. The user can press 
the "bad computer" button, which then adjusts the weight 
and/or the J-measure for that rule governing the last chord 
that was produced. That makes it less likely that the rule 
will be used subsequently. The opposite is also true - a 
particularly good sound can be made more likely to recur by 
initiating the "good computer" button. 

The system operates by firing rules which have 
certain weights. The weights are initially assigned by the 
learning algorithm, based on how well the' rules perform 
(rules which are able to fire frequently or which are right 
more of the time are given higher weights) , 
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In addition to input through the MIDI keyboard, the 
user is also given access to two buttons. These buttons are 
labelled "good computer" and "bad computer 11 , and are pressed 
when the user either likes or dislikes what the system is 
doing . 

At any point, the user can press one of the buttons. 
These buttons affect the weights of the rules which fired to 
produce the notes generated by the system immediately 
preceding the button press. 

When the "good computer" button is pressed, all the 
rules which predicted (voted for) the system's actual output 
have their weights increased. The weights can either be 
increased by a fixed value (for example, each rule which 
fired might have its weight increased by 0.01), or they can 
be increased by a fixed fraction (for example, each rule 
which fired might have its weight multiplied by 1.01) . 

Similarly, the "bad computer" button decreases the 
weights of all rules which contributed to the output which 
the user did not like. 

For example, assume for a given times tep the 
following rules fire: 

1 . If A then B (weight 0.50) 

2. If A then C (weight 0.40) 
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And let's say that the system picked B as the output 
of the system. 

If the user hit the "good computer" button, we would 
increase the weight for rule 1 (say, to 0.51), since the 
5 user liked what that rule predicted. 

If the user hit the "bad computer" button, we would 
decrease the weight for rule 1 (say, to 0.49), so that the 
system is less likely in the future to do what the user 
didn't like. 

10 Subsumption pruning takes place during rule 

generation, which is when the system applies a series of 
rule bases to a melody to fill in a figured bass (Fig. 9) . 
When a rule base is used to infer a RHS value during rule 
generation, each rule in the rule base is checked in order 

15 of decreasing J-measure (step 1060a) . If a rule's 

dependence value f is zero and all of the attributes on its 
left-hand side are known, the rule can fire, adding its 
weight to the weight of the RHS value which it predicts. 
After all rules have had a chance to fire, the result is an 

20 array of weights for all possible values of the RHS 
attribute. The weights of all rules inferencing a 
particular RHS value are accumulated to produce the weight 
of that RHS value (step 1060b) . 
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Resolving conflicts is necessary when two or more 
rules fire and inference a number of different RHS values 
(step 1060c) . After exponentiating and normalizing the 
accumulated weights for the different RHS values to produce 
probabilities for each value, the system chooses one of 
these values at random. The system does not have to choose 
the answer probabilistically. If it does, it chooses the 
answer randomly, based on the probabilities generated by 
exponentiating the weights for the possible RHS values. 
However, we could also simply choose the most likely answer. 

- Summation of Rule Weights 
When a rulebase is used to infer a RHS value, each rule 
in the rulebase is checked in order of decreasing rule 
j-measure. A rule can fire if it has not been marked 
dependent (see the next section on independence pruning) and 
all the attributes on its LHS are known. When a rule fires, 
its weight is added to the weight of the RHS value which it 
predicts. After all rules have had a chance to fire, the 
result is an array of weights for all possible values of the 
RHS attribute. 

Independence Pruning in Real Time 



- 48 - 



WO 97/35299 



PCT/US97/04636 



As explained in the section above on generation of 
dependence data, all rules which fire for a given LHS should 
be independent. However, the inventors realized that 
rulebases cannot be pruned ahead of time to remove rules 
without losing information. 

The inventor's solution to this dilemma is to keep 
track of which rules are dependent on other rules, and only 
allow rules which are still independent to fire. This 
technique is described below. 

Start by allocating and zeroing an array F , where f k 
is zero if rule R t is allowed to fire. Then for each rule 
R A in order of decreasing J-measure, 

1. If fi is non-zero, the rule is not allowed to fire. 
Skip to the next rule. 

2. If the rule can't fire, one of the attributes on the 
LHS of the rule is either unknown in the input data or does 
not have the right value to match the input data, skip to 
the next rule.????? 

3. The rule can fire. Add its weight to the weight for 
the RHS value it predicts. 

4 . For each rule Rj in the list of rules dependent with 
R if set the corresponding fj non-zero. 
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This technique is very fast, since it requires only 
array lookups and does no complex calculations. In fact, it 
is faster than using the same rulebase without dependency 
information, since if a rule is forbidden from firing the 
program does not spend time determining if the rule is 
allowed to fire. (With no dependency information, all rules 
are checked to see if they can fire.) 

4.3 Resolution of Conflicts Between Rules Which Fire 

If all rules which fire on a given example inference 
the same RHS value, the result of the inference is clear. 
But if two or more rules fire and inference a number of 
different RHS values, one of two algorithms must be used to 
resolve the conflict. In either case, the weights of all 
rules inferencing a given RHS are accumulated to produce the 
weight of that RHS. 

The simpler algorithm is termed "best-only." The RHS 
with the highest weight is always chosen. This is the most 
correct method from the standpoint of probability theory. 
However, the inventors realized that this tends to lead to 
monotonous music, since a given melody will always be 
harmonized in the exact same fashion. 
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This problem led to the development of a second 
algorithm. 

The other option is to randomly select between the 
possible RHS values. The accumulated weights for the RHS 
values are exponentiated and normalized to produce 
probabilities for each value. The RHS value to be used is 
chosen randomly based on these probabilities. It is 
important to note that the algorithm only chooses between 
values which had rules fire, not all possible values for the 
RHS attribute. Otherwise, there would always be a non-zero 
probability of picking any RHS value, even if no rules fired 
for that value. 

- 4.4 What If No Rules Fire? 
If no rules for a given rulebase fire, there are two 
possibilities. If it is not the last part of a series of 
segmented rulebases, the next segmented rulebase will be 
given a chance to fire. If the rulebase is the last in the 
series, or is not part of a series of segmented rulebases, 
the RHS value is set to the most likely value of the RHS 
attribute based on the attribute's prior probability 
distribution. This is equivalent to classifying the RHS 
attribute with a zeroth-order Bayesian classifier. 
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This problem can be avoided by training a first-order 
Bayesian classifier and using it as the last segment in a 
series of rulebases for a given RHS attribute. (For 
example, basing the current chord function only on the 
i current melody pitch and setting both the minimum 

probability for a rule and the minimum rule J-mea'sure to 
zero.) Since the first-order classifier will always have 
exactly one rule which fires, more information will be used 
to pick the RHS value than if no rules fired at all. 

) - Conversion to MIDI 

The output of harmonization is either saved in a 
MIDI file or played on a MIDI synthesizer, so conversion 
from figured bass back to MIDI is necessary (Fig. 10). MIDI 
data is produced for each timestep as follows. First, using 

5 the table of common functions and the voice position fields, 
the system determines for the chord which voices should play 
which pitches (step 1070a) . Starting just below the melody 
note, which is known because it was used as the input to 
harmonization, the system then searches, once for each 

0 remaining voice, for an unplayed note matching that voice's 
pitch (step 1070b) . Lastly, using MIDI code, the system 
indicates the notes found (step 1070c) , the delays equal to 
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each note's duration (step 1070d) , and corresponding note 
terminations (step 1070e) . 

Given the timestep below, for example, the system 
uses the table of common functions to determine that the 
5 "iii" chord has the pitches {E, G, B} . Based on the 

positions {12, Tl, Al, SO} with the soprano pitch agreeing 
with the melody field, the voices play pitches {B, G, G, E}, 
respectively. If the melody note were at octave 5, the MIDI 
conversion would turn on the notes {E5, G4 , G3 , B2 } . In 
10 either case, the system would encode a delay and a 

termination corresponding to a duration of one-eighth note. 
MEL FUNC IN TP AP SP DUR ACC 

E iii 12 Tl Al SO 1 un 

15 - Rulebases and Results 

In the following discussion of the development of 
sets of rulebases, results from these sets of rule bases are 
analyzed and contrasted with each other. When rulebases are 
printed in a table, the columns have the following meanings. 
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RHS 
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Attributes 




Rules 
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The number 


Signi- 


attribute 


present on 


maximum 


of rules 


ficant 


present 


the LHS of 


number of 


in the 


features 


on the 


the rule 


terms 


rule base . 


of the 


RHS of 


base . 


allowed on 




rule 


the rule 


Rules must 


the LHS of 




base . 


base . 


contain 
any 

attributes 
in bold, 
and may 
contain 
the other 
attri- 
butes . 


a rule. 







10 Unless otherwise noted, all rules should be correct 

at least 50% of the times they fire and should have a J- 
measure of at least 0.001. The rules discussed below were 
trained from an example set of 15 Bach harmonized chorales, 
which produced 818 examples by beat-based conversion and 834 

15 examples by chord-based conversion. 

The first attempt at generating harmony rules used 
no rule base segmentation, filtering, or priming. The 
resulting rule base, called Simplel, was trained from 
examples using beat-based conversion. 
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RHS 

Attribute 


LHS 

Attributes 


Max Order 


Number of 
Rules 


Notes 


FunctionO 


Functionl , 

Melodyl, 

MelodyO 


3 


105 





This initial rule base had a number of limitations. 



5 Of its 105 rules, 3 3 do not use the current melody note or 
the previous function, which lead to unresolved dissonances 
in the harmony. For example, if the current melody note was 
F- sharp and the previous function was a V7 chord, the 
following rule led the rule base to play a C Major chord. 

10 12. IF Functionl V7 THEN FunctionO I : 0.566 0.343 0.030 

The C Major chord sounds very dissonant against the F- sharp 
in the melody. 

To correct the problems in the first rule base 
Simplel, all rules which did not use both the current 
15 function and previous melody note were filtered out, 
producing a new rule base Simple2. 



RHS 

Attribute 


LHS 

Attributes 


Max Order 


Number of 
Rules 


Notes 


FunctionO 


Functionl, 

Melodyl, 

MelodyO 


3 


72 
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However, this smaller rule base frequently failed to 
fire on its input. This led to the following harmonization 
of the first phrase of "Hark, the Herald Angels Sing:" 



Melody Chord 



Rules Fired 



10 



5 



G4 
C5 
C5 
B4 
C5 
E5 
E5 
D5 



I 
I 
I 
I 
I 
I 
I 
V 



0 
0 
0 
0 
0 
2 
2 
2 



Too much information had been lost, so no rules were 
fired for over half the timesteps, producing an extremely 
15 dull harmony. The smaller rule base sounded worse, because 
dissonances were created when no rules fired and the C-Major 
chord picked by the Bayes classifier of order zero was 
played against notes such as F and B. 



20 recognized with respect to the first two rule bases lay in 
segmenting the learned harmony rules into three rule bases, 
together called Major4 and listed in the table below. These 
rule bases were the first to be used in real time to 
accompany a musician. The musician played only the melody 



The solution to the problems that the inventors 
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note and the program responded with the other three voices a 
fraction of a second later. 

The first rule base contained the best rules, used 
in the Simple2 set. If no rules from that set fired, the 
5 second rule base tried to fire rules which used at least the 
current melody note. As mentioned above with respect to 
segmentation, this method allowed the better rules a chance 
to fire without being overwhelmed by rules using less 
significant information, while preserving all of the 

10 information contained in the full rule base. 

If no rules fired in any of the three initial rule 
bases, which happened about 25% of the time, a first-order 
Bayesian classifier would determine the current function 
based on the current melody note. This ensured that the 

15 chord played would be at least consonant with the melody 
note. 

These rules worked well enough that additional rule 
bases were generated to determine the positions of the bass, 
alto, and tenor voices so that the harmonized melody could 
be converted back into MIDI data and played, as described 
above. Bayesian classifiers were not needed in addition to 
these rule bases, because (l) the generated rules spanned a 
much larger portion of the input space, i.e. . only rarely 
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did no rule fire, and (2) because an error in a single voice 



position is much less noticeable than a bad chord function. 



RHS 

Attribute 


LHS 

Attributes 


Max Order 


Number of 
Rules 


Notes 


FunctionO 


Functionl, 

Melodyl, 

MelodyO 


3 


172 


First of 
four rule 
bases 
used to 
predict 
harmony . 


FunctionO 


Melodyl , 
MelodyO 


3 


34 




FunctionO 


Functionl , 
Melodyl 


3 


37* 




FunctionO 


MelodyO 


1 


8 


First- 
order 
Bayesian 
classi- 
fier. 


Inver- 


Functionl, 
Tnver - 

sionl , 
FunctionO 


3 


145 




AltoO 


Functionl, 

Altol, 

FunctionO, 

Inver- 

sionO 


3 


472 




TenorO 


Tenorl , 
FunctionO, 
Inver- 
sionO , 
AltoO 


3 


341 
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Some of the significant rules in these rule bases 
included the following. 

The first rule is from the first FunctionO rule 

base . 

5 1. IF MelodyO E THEN FunctionO I 0.83 0.89 0 0601 

AND Functionl V 

This transition, from G Major to C Major, is the 
strongest cadence or ending in classical harmony. 

3. IF MelodyO F THEN FunctionO IV 0.98 3 12 0 0499 

10 AND Functionl V 

This is another common transition, from G Major to F 

Major. 

The following rule is from the InversionO rule base. 

1- IF Functionl V THEN InversionO II 0.98 1.59 0 0255 

15 AND FunctionO IV 

Combined with rule 3 above, this rule places the 
function V to function IV transition in first inversion. 

3. IF Functionl V THEN InversionO 10 0.86 0.20 0 0179 

AND FunctionO I 

20 Combined with rule 1 above, this places the function 

V to function I cadence in root position, which is the 
strongest position for an ending chord. 

26 * IF FunctionO vii07 THEN InversionO II 0.53 0.17 0.0098 

This rule places diminished 7th chords in first 
25 inversion, where they are placed in classical harmony. This 
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rule has a lower J-measure than the other rules because 
diminished 7th chords do not appear very often, which 
creates a low value for p(y) . 

With the "best -only" method turned off as described 
5 above, the system was able to produce different harmonies 
for a given melody by randomly choosing among possible RHS 
values. For example, the melody C-A-B-G-D-C could be 
harmonized as follows. 



c 


Major 


10 


AO 


T2 


C5 


I 


{ C3 


G3 


C4 


C5 


D 


Major 


10 


A2 


TO 


A5 


: V/V 


{ D3 


D4 


A4 


A5 


G 


Major 


10 


Al 


T2 


B5 


: V 


{ G3 


D4 


B4 


B5 


G 


Major 


11 


A2 


TO 


G5 


: V 


{ B3 


G4 


D5 


G5 


G 


Major 


10 


Al 


T2 


D5 


: V 


{ G3 


D4 


B4 


D5 


C 


Major 


10 


AO 


T2 


E5 


: I 


{ C4 


G4 


C5 


E5 



15 Alternatively, the melody could be harmonized as shown 
below. 



C Major 10 

D Major 10 

G V7 10 

20 C Major 10 



AO T2 C5 : I 

A2 TO A5 : V/V 

A3 T3 B5 : V7 

AO Tl G5 : I 



{ C3 G3 C4 C5 } 

{ D3 D4 A4 A5 } 

{ G3 F4 F5 B5 } 

{ C4 E4 C5 G5 } 
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B dim7 II A3 Tl D5 : vii07 { D3 D4 G#4 D5 } 

A Minor 10 Al TO C5 : vi { A2 A3 C4 C5 } 

The two harmonizations are quite different: in the 
six-note melody above, there are three places where the 
5 program has a choice between two functions for a given 
chord. 

Another piece harmonized by these rule bases, the 
first phrase of "Hark! the Herald Angels Sing" shown in Fig. 
11, has a generally high-quality sound - there are no 

10 unresolved dissonances. However, the voice- leading in the 
piece is poor in places. The third chord, a C Major chord, 
has notes {C, C, C, G} . The third note of the chord, E, is 
absent, leading to a hollow sound. This problem was 
addressed in the next set of rule bases, called Major4a and 

15 discussed below. 

In an attempt to correct the voice leading problems 
of the Major4 rule base, a rule base which determined the 
soprano voice position was added to the set of rule bases. 
Since the current function and melody pitch uniquely 

20 determine the soprano voice position, the generated rule 

base covered the entire input domain and was always correct. 
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The soprano voice position was added to the possible 
LHS attributes for the rule bases for the other voice 
positions. This permitted rules for the tenor which would 
allow the tenor to fill in a missing chord pitch. The tenor 
5 rules were no longer forced to include the chord position. 





The 


addition of 


the 


soprano voice 


allows rules such 




following. 










2 . 


IF 


SopranoO 
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THEN TenorO T2 


: 0.888 1.024 

0 . 132 
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AND 
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AO 
10 








6 . 


IF 
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THEN TenorO Tl 


: 0.901 1.239 

0 .079 


15 




AND 
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InversionO 
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IF 
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S2 


THEN TenorO T3 


: 0.634 1.326 

0 .070 


20 




AND 
AND 
AND 
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TO 












These rules 


show the tenor 


rule base filling 



chord pitches which are not present in the other rule bases. 
The very high accuracy of the first two- rules (88.8% and 
90.1%) indicates that it is important to fill out a chord's 
25 pitches. 

The number of rules is then reduced by subsumption 
pruning of the rulebases, resulting in the Major4a set shown 
in the table below. This pruning removed from 5% to 30% of 
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the rules from any given rule base without affecting its 
classification accuracy or input domain. 





RHS Attribute 


LHS 

Attributes 


Max Order 


Number of 
Rules 


Notes 




FunctionO 


Functionl, 

Melodyl, 

MelodyO 


3 


124 




5 


FunctionO 


Melodyl, 
MelodyO 


3 


32 






FunctionO 


Functionl, 
Melodyl 


3 


26 






FunctionO 


MelodyO 


1 


8 


First -order 
Bayesian 
classifier . 




Soprano 


MelodyO , 
FunctionO 


2 


60 


Direct 
equivalence 
between LHS 
and RHS . 




InversionO 


Functionl, 
Inversionl, 
FunctionO, 
SopranoO 


4 


133 




10 


AltoO 


Functionl , 
Altol, 
FunctionO , 
InversionO, 
SopranoO 


4 


309 






TenorO 


Tenorl, 
FunctionO , 
InversionO, 
AltoO, 
SopranoO 


4 


434 

- 





Fig. 12 shows the harmony for "Hark! The Herald 



Angels Sing" generated by the new rules. The third chord, 
which used the voice arrangement {C,C,C,G} under Major4, 
uses {C,G,E,G} under Major4a and contains all three pitches 
present in the C Major chord. Furthermore, the new rules 
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doubled the G note, as is proper for a chord present in 
second inversion. 

Despite the progress in voice-leading, the Major4a 
rules still had limitations. For instance, the rules 
5 referred back in time only to the previous chord, and did 
not use information about the accent on the current chord.- 
This meant that the rule base could not predict when a piece 
of music was ending, and thus often fumbled the final 
cadence. An example of this problem is shown in Fig. 13 in 
10 the harmony produced for "Happy Birthday." The harmony ends 
on a "vi" or "A Minor" chord, which, being a minor chord, 
lends a sad feel to the end of the piece. This is not an 
appropriate way to end a piece written in a major key. 

The Major7a set of rule bases, listed below, was 
15 allowed to use more information about the accents of current 
and previous chords. "FunctionLA" stands for the function 
of the last chord which started on an accented beat. 
"FunctionLB" and " InversionLB" represent the function and 
inversion, respectively, of the last chord which started at 
20 the beginning of any beat. "AccentO" means the accent on 
the current chord. "Functionl" still stands for the 
function of the immediately preceding chord. 
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With the Bach chorales used as input, either 
FunctionAB or FunctionLB did not match a common function 14% 
of the time. The method could not find a match for 
Functionl in 25% of the examples. Since unmatched functions 
5 typically indicate that an ornament is present, this result 
confirms that ornaments occur more frequently in the middle 
of beats. 

Rules were required to be correct at least 3 0% of 
the time they fired, which was lower than the 50% required 
10 by previous sets of rule bases. However, the largest prior 
probability for FunctionO was 24%, so a rule which was 
correct 3 0% of the time still provided useful information. 
All rule bases were also subsumption pruned. 
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RHS Attribute 


LHS 

Attributes 


Max Order 


Number of 
Rules 


Notes 




FunctionO 


FunctionLA, 

FunctionLB, 

Functionl, 

Melodyl, 

AccentO , 

MelodyO 


5 


175 


First of 
four 

segments of 
FunctionO 
rules . 


- 


FunctionO 


( FunctionLA 
and/ or 
FunctionLB) , 
Melodyl , 
AccentO , 
MelodyO 


5 


282 








Melodyl , 
AccentO , 
MelodyO 


5 


83 





5 


FunctionO 


FunctionLA, 
FunctionLB, 
Funct ionl , 
Melodyl, 
AccentO 


5 


361 






SopranoO 


FunctionO , 
MelodyO 


2 


60 


Direct 
equivalence 
between LHS 
and RHS. 




InversionO 


FunctionLB , 
Functionl , 
Inversionl , 
FunctionO , 
SopranoO 


5 


332 


First of two 
segments of 
InversionO 
rules . 




InversionO 


FunctionLB, 
Functionl , 
Inversionl , 
SopranoO 


5 


287 






AltoO 


Functionl, 

AltOl, 

FunctionO, 

InversionO, 
SopranoO 


5 


820 




10 


TenorO 


Tenorl , 
FunctionO , 
InversionO , 
AltoO, 
SopranoO 


5 


815 
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10 



15 



20 



Rules had more possible LHS attributes and higher 
order rules were permitted, so enough rules were generated 
that at least one rule would fire for each desired RHS 
attribute in almost all cases. Therefore, a Bayesian 
classifier was not needed as a safety net for determining 
the chord function. 

The script for determining the major7 follows. 
Lines which start with ; are comments. 
; Read examples from the example list 

load exlist major7 from major7.el 

Set defaults 

/ 

At most 5 clauses on "IF" side of a rule 
default rule order 5 

Unless otherwise specified, learn using the "major7 
example list we just read in 
default exlist major7 



Learn up to 204 8 rules at a time 
25 default maxrules 2048 

; Rules must be right at least 30% of the time 

default mincorrect 0.3 

30 ; Rules must have a J-measure >= 0.001 

default minpriority 0.001 



35 ; Extract and save attributes 

copy attrbase attr7 from major7 
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save attr7 to attr7.att 



Learn rules for HarmonyO 

learn harm7_2 { 

; These attributes CAN appear on the left-hand side 

lhs MelodyO 

lhs Melodyl 

lhs Functionl 

lhs FunctionLB 

lhs FunctionLA 

lhs AccentO 

; This is what we want to predict 
rhs FunctionO 

} 

• Now we want to segment the harmony rules into 3 

different 

sets, based on what attributes they contain. 



• Ruleset #3 - doesn't use current melody 

Copy the full set of rules 
copy rulebase harm7_3 from harm7_2 
; Remove any rules which use MelodyO 

filter harm7_3 never MelodyO 
; Do subsumption pruning 

prune harm7_3 

Save the rulebase 
save harm7_3 to harm7_3 . rul 

And free up its memory 
free harm7_3 

Rulesets #1,2 - use current melody and last 
functions 

Now remove all the rules which ended up in harm7_3 
filter harm7_2 always MelodyO 
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10 



; And resize the rulebase (this frees up the memory 

which 

; was used by the rules we just filtered out) 

resize harm 7 2 



; Ruleset #1 - use either Functionl, FunctionLB, or 

FunctionLA 



; In order to handle the "OR" in the statement above, 

we need 

; to make three sub-rulebases - each contains rules 

which use 

15 ; one of the Function attributes. 

copy rulebase h71a from harm7_2 

filter h71a always Functionl 

prune h71a 

resize h7ia 
20 copy rulebase h71b from harm7_2 

filter h71b always FunctionLB 

prune h71b 

resize h71b 

copy rulebase h71c from harm7_2 
25 filter h71c always FunctionLA 
prune h71c 
resize h71c 

; Now we combine the three sub-rulebases into one big 

3 0 rulebase. 

combine h71a and h7lb into h71d 
free h71a 
free h7lb 

combine h71c and h71d into harm7_l 

3 5 free h71c 

free h71d 

; Once they're combined, we can subsumption-prune the 

result . 

4 0 prune harm7_l 

save harm7_l to harm7_l.rul 
free harm7_i 

; Ruleset #2 - doesn't use any functions 

45 filter harm7 2 never Functionl 
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filter harm7_2 never FunctionLB 
filter harm7_2 never FunctionLA 
prune hartn7_2 

save harm7_2 to harm7_2.rul 
5 free harm7 2 



Learn rules for SopranoO (should do perfectly 
"10 there' s 

a 1:1 mapping between FunctionO +MelodyO and 
SopranoO) 

learn sopr7_l { 
15 ruleorder 2 

mincorrect 0 . 2 

minpriority 0.000001 

lhs MelodyO 

lhs FunctionO 
20 rhs SopranoO 

filter sopr7_l always MelodyO 
filter sopr7_l always FunctionO 
save sopr7_l to sopr7_l.rul 
25 free sopr7_l 



Learn rules for InversionO 



30 



learn invr7_2 { 
lhs FunctionO 
lhs SopranoO 
lhs Inversionl 

3 5 lhs Functionl . 

lhs InversionLB 
lhs Funct ionLB 
lhs AccentO 
rhs InversionO 

40 } 

; Ruleset #1 - use current function 

copy rulebase invr7_l from invr7_2 
filter invr7_l always FunctionO 

4 5 prune invr7_l 
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10 



30 



save invr7_l to invr7_l.rul 
free invr7_l 

; Ruleset #2 - don't use current function 

filter invr7_2 never FunctionO 
prune invr7_2 

save invr7_2 to invr7_2 . rul 
free invr7 2 



Learn rules for AltoO 



learn alto7_l { 
15 lhs FunctionO 

lhs SopranoO 

lhs InversionO 

lhs Functionl 

lhs Altol 
20 lhs AccentO 

rhs AltoO 

} 

prune alto7_l 

save alto7_l to alto7_l.rul 
25 free alto7 1 



Learn rules for TenorO 



learn tenr7_l { 
lhs FunctionO 
lhs SopranoO 
lhs AltoO 

3 5 lhs InversionO 

lhs Functionl 
lhs Tenorl 
rhs TenorO 

} 

4 0 prune tenr7_l 

save tenr7_l to tenr7_l.rul 
free tenr7__l 

We're done with this section of the learning, so 
45 exit this script. 
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end 

This Major7a set of rule bases produces the harmony 
for "Happy Birthday" shown in Fig. 14. Unlike Major4a, 
Major7a directs that the piece should end on a "I" or C 
5 Major chord, which is a more solid ending for a piece in a 
major key. 

The Major7b set of rule bases, shown in the table 
below, is identical to the Major7a set except for the 
addition of dependency data for real time independence 
10 pruning. The number of dependent rule pairs for each rule 
base is shown in the table. 



RHS Attribute 


LHS 

Attributes 


Number of 
Rules 


Number of 
Dependent 
Rules 


Average 
Pairs Per 
Rule 


FunctionO 


FunctionLA, 
FunctionLB, 
Functionl , 
Melodyl, 
AccentO , 
MelodyO 


175 


175 


1 . 0 


FunctionO 


(FunctionLA 
and/ or 
PunctionLB) , 
Melodyl , 
AccentO , 
MelodyO 


282 


249 


0 . 9 


FunctionO 


Melodyl, 
AccentO , 
MelodyO 


83 


32 


0.4 


FunctionO 


FunctionLA, 
FunctionLB, 
Functionl , 
Melodyl, 
AccentO 


361 


553 


1.5 
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SopranoO 


FunctionO , 
MelodyO 


60 


0 


0 . 0 


InversionO 


FunctionLB, 
Functionl , 
Inversionl , 
FunctionO , 
SopranoO 


332 


694 


2 . 1 


InversionO 


FunctionLB, 
Functionl, 
Inversionl , 
SooranoO 


287 


597 


2 . 1 


AltoO 


Functionl, 
Altol, 
FunctionO, 
InversionO , 
SopranoO 


820 


1992 


2.4 


TenorO 


Tenorl , 
FunctionO, 
InversionO , 
AltoO, 
SopranoO 


815 


2868 


3.5 



The position-oriented rule bases, which have more 
LHS attributes which take only a few values, end up with 
higher numbers of dependent rule pairs . This leads to 
situations such as the following. * If the TenorO rule base 
10 contains the rule 



IF SopranoO = S2 THEN TenorO * Tl 



then the TenorO rule base is likely to contain one or more 
of the following rules 



IF SopranoO » S2 THEN TenorO - Tl 

15 AND Tenorl » TO 
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IF SopranoO = S2 THEN TenorO = Tl 

AND Tenon = Tl 

IF SopranoO = S2 THEN TenorO = Tl 
AND Tenorl = T2 

IF SopranoO = S2 THEN TenorO = Tl 
AND Tenorl = T3 

because a subset of examples with a specified value for 
Tenorl has a sufficiently large number of samples to force 
up the J-measure for rules with that Tenorl value on the 
LHS. 

The addition of real time independence pruning 
speeds up harmonization because fewer rules in each rule 
base need to be checked to see if they can fire. However, 
the harmony generated by the newer rule bases does not 
differ significantly from that of the Major7a rule bases. 

The following script is used: 

; MAJ0R7B . INP - generates dependence info for major7 rules 
• We did this as a separate script so I could look at the 
intermediate 

; steps - there's no reason we couldn't do it m the same 
script that 

; we learned the rules in. 



; Load our examples and attributes. 

load exlist m7 from major7.el 
copy attrbase a7 from m7 
default attrbase a7 

; Now we load in each rulebase and generate its dependency 
information . 
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; Load the rulebase 

load rulebase r from r\harm7_l . rul 

; Generate its dependency information 

gendep r with m7 0.5 

; And save it 

save r to harm 7_lb. rul 

; Then free up the memory it was using, 
free r 

load rulebase r from r\harm7_2 . rul 
gendep r with m7 0.5 
save r to harm7_2b.rul 
free r 

load rulebase r from r\harm7_3 . rul 
gendep r with m7 0.5 
save r to harm7_3b.rul 
free r 

load rulebase r from r\harm7_4 . rul 
gendep r with m7 0.5 
save r to harm7_4b.rul 
free r 

load rulebase r from r\invr7_l . rul 
gendep r with m7 0.5 
save r to invr7_lb.rul 
free r 

load rulebase r from r\invr7_2 . rul 
gendep r with m7 0.5 
save r to invr7_2b.rul 
free r 

load rulebase r from r\alto7_l . rul 
gendep r with m7 0.5 
save r to alto7_lb.rul 
free r 

load rulebase r from r\tenr7_l . rul 
gendep r with m7 0.5 
save r to tenr7_lb.rul 
free r 
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end 

- File Format 

The following describes a specification of a 
5 preferred data file format for transmitting information 

about examples and rules among different applications. The 
format allows for expansion of the specification while still 
permitting older applications to read newer and expanded 
data files. Any application which implements the required 
10 portions of the specification is able to read and use those 
portions of any data file written using any version of the 
specification. 

The preferred file extension is " . IPR," which stands 
for Itrule Portable Rule ("IPR") file. 
15 An IPR file includes ASCII text. The first ten 

characters of an IPR file should be " # I PRSTART# " which 
permits application readers to detect and reject easily 
files which are not IPR files. The file terminates with the 
text string 11 #IPREND#" followed by an End-of-File ("EOF") 
20 character, which is OxlA in hexadecimal notation. Lines can 
terminate with any combination of carriage -return (OxOD) and 
line feed (OxOA) characters. The line length limit is 16384 
characters . 
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IPR files can consist of any number of sections - 
for example, an IPR file with zero sections is meaningless, 
but permissible. All identifiers and variable names are 
case-insensitive. Identifiers and variable names should 
begin with a letter, i.e. , A to Z, and should not contain 
space characters or any of the following characters: 

{} = ,"<)> 

Identifiers and variable names can be up to 31 characters 
long. Values can be up to 255 characters long. 

Each section of the data file has the following 



form. 



SECTIONTYPE "{ 

. . .data for section. . . 



15 Th * " SECTIONTYPE " identifier is not required to be 

on the same line as the open brace and no space is required 
between the identifier and the open brace. 

Under the specification, a program which does not 
recognize a section type should ignore it. Sections can be 
nested, e.g. , a "RULE" section can be nested inside a 
" RULEBASE" section. A nested section is referred to as a 
"subsection." Within a section, all variables should come 
first, followed by any subsections. 
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Comment notation is similar to that of the 
programming language C++. Single-line comments begin with 
two slashes "//" and extend to the end of the line, as shown 
below. 

5 // This is a comment 

Comments with multiple lines, such as the sample comment 
below, begin with slash-star "/*" and end with star-slash 
" * / " . 

/* This is a comment 
10 which can extend 

over multiple lines */ 

Any text denoted a comment should be ignored by programs. 
Variable assignments have the following form. 

variable = value 

15 A value containing spaces or tabs should be enclosed in 
double -quotes, as shown below. 

variable = "multi word value" 
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Spaces between the variable, equals sign " = " , and the value 
are optional. A program reading an assignment should be 
able to understand the assignment with or without the 
spaces . 

Some variables are optional and can be absent from 
an IPR file - a program is not required to be able to read 
or write these variables. A program encountering a variable 
unknown to it should be able to pass over that variable 
without disruption. 

A required variable is indicated by a denotation 
"(required)" which follows the variable's definition. All 
reader applications and writer applications should process 
these variables. 

Variables have assigned types which follow their 
definitions: "string" denotes an ASCII string, "integer" 
indicates a 4 -byte signed integer, and "float" signifies a 
floating point number. 

Some section types are pre-defined. A "RULEBASE" 
section is used to store lists of rules and consists of a 
series of variables followed by a series of rule sections, 
as shown below. 



RULEBASE { 
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// variables: 
NAME = string 
COUNT = integer 
ATTRIBUTES FROM = string 

DEPEND ENCYCOUNT = integer (^Enumerates the size of 
dependency table * ) 



// list of rules: 
RULE { 

. . . rule data. . . 

RULE { 

. . . rule data. . . 

} 



// realtime dependency table 
DEPENDENCYTABLE { 

. . .dependency data. . . 



In the " RULEBASE " section, the variable "NAME 
(string, required)" has the rule base's name, which can be 
up to 256 characters in length. A variable "DEPENDENCYCOUNT 
(integer, optional)" indicates the number of elements in the 
real-time dependency pruning table and should be present if 
the " DEPENDENCYTABLE " subsection is present. The number of 
rules in the rule base is stored by the variable "COUNT 
(integer, required) . " 

An attribute data base, in terms of which the rule 
base is defined, should precede the rule base in the IPR 



- 80 - 



WO 97/35299 



PCTAJS97/04636 



file and is indicated by variable "ATTRIBUTES FROM (string, 
required) . " 

Two sections contained in a "RULEBASE" section are 
"DEPENDENCYTABLE (optional)" and " RULE" (required)." The 
" DEPENDENCYTABLE " section contains real-time dependency 
information for the rule base and is stored as a series of 
integers separated by spaces. The "RULE" section stores a 
single rule and is contained in a " RULEBASE" section. 

A "RULE" section has the structure shown below. 



RULE { 

PRIORITY = float 
WEIGHT = float 
J -MEASURE = float 
LITTLE- J = float 
P (FIRE) = float 
P (CORRECT) = float 
DEPENDOFFS = integer 



IF { 

// permissible if clauses:, 
■[attr = value} 
|attr <> value} 
{attr > value J 
''attr < value) 
'attr >= value) 
attr <= value} 

} 

IFOR {} 
IF AND {} 

THEN j 

{attr = value | weight} 
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{attr = value | weight} 

THENDISTR { 

{attr | weightl weight2 weight3 . . 

} 



An example of an "IF" clause is shown below. 



// if (al=vl and a2=v2 and (a3=v3 or a4=v4)) 
0 IF AND { 

T al = vl 



15 



a2 = v2 
IFOR 



a3 = v3^ 
a4 = v4 



} 



In a "RULE" section, the variable "PRIORITY (float, 
optional)" indicates the rule's priority, in arbitrary 

20 units. Rule weight is signified by the variable "WEIGHT 

(float, optional)" which stores the logarithm of the rule's 
transition probability. The variables "J-MEASURE (float, 
required)" and "LITTLE- J (float, optional)" contain the 
rule's J-measure and j -measure, respectively. The 

25 probability, based on the training examples, that the rule 
will be able to fire is indicated in the variable 11 P (FIRE) 
(float, optional)." Related variable "P(CORRECT) (float, 
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optional) " represents the probability, again based on the 
training examples, that the rule, if able to fire, will be 
correct. If a dependency table is used, the variable 
"DEPENDOFFS (integer, optional) " shows the offset position, 
in the realtime dependency table, of the rule's dependency 
information. 

Subsection "IF (required)" has a standard left-hand 
side with "attribute = value" pairs and should not have 
nested boolean expressions. The attribute and value should 
conform to the specifications for variables. 

Subsection "IFAND (optional)" is equivalent to 
subsection "IF." Subsection "IFOR (optional)" returns a 
boolean value of "TRUE" if one or more of its "attribute = 
value" pairs matches the input data. Subsections "IFAND" 
and "IFOR" can be nested within each other. 

The subsection "THEN (required) " has a standard 
right-hand side with "attribute = value | weight " sets. The 
"weight" field, which is optional, represents the fraction 
of the total rule weight, indicated by the WEIGHT variable 
discussed above, which should be added to the logarithmic 
probability for the RHS value. The "weight" fields are not 
required to add up to 1.0. An omitted "weight" field is 
treated as a "weight" field of 1.0. As mentioned above, the 
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attribute and value should conform to the specifications for 
variables. Distribution rules can be represented by a 
"THEN" subsection which has one triplet for each possible 
RHS value or by a "THENDISTR (optional)" subsection which 
specifies an attribute and lists the weights for each value 
of that attribute in order. 

As mentioned above, each rule base is defined in 
terms of an attribute base. An "ATTRBASE" section, which 
has the form shown below, stores an attribute base, i .e . , a 
series of attributes, just as a "RULEBASE" stores a series 
of rules. 

ATTRBASE { 

// variables: 
NAME = string 
COUNT = integer 

// list of attributes: 
ATTRIBUTE { 

. . .attribute data . . . 

} 

ATTRIBUTE { 

. . .attribute data. . . 

} 

} 

The "NAME (string, required) " variable in the 
attribute base stores the attribute base's name, which can 
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be up to 256 characters in length. The number of attributes 
in the attribute base is represented by COUNT (integer, 
required) . 

The "ATTRIBUTE (required) » subsection has the 
structure shown below. 



ATTRIBUTE { 

// variables: 
NAME = string 
COUNT = integer 
UNKNOWN = float 



// values 

VALUES { 

Value probability 
value probability 
'value probability 




The variables of the "ATTRIBUTE" subsection include 
the "NAME (string, required)" variable which stores an 
attribute name of up to 256 characters in length and the 
" COUNT (integer, required)" variable which represents the 
number of values for the attribute. Another variable 
"UNKNOWN (float, optional)" indicates the fraction of the 
attribute's values that are unknown. A list of values and a 
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probability for each value is stored by the "VALUES 
(required)" variable. 

The " RB AS EL I ST" subsection is a section containing a 
list of rule bases and has the structure shown below. 



RBASELIST { 

// variables: 

NAME = string 

COUNT = integer 

// filename for attrbase 

ATTRBASE = string 

// rulebases in order 
RBLIST { 



(name 
(name 



flagl flag2 
flagl flag2 



Like other sections, the " RBASELIST" section has a 
"NAME (string, required)" variable and a "COUNT (integer, 
required}" variable. The "COUNT" variable represents the 
number of rule bases in the list. The common attribute base 
for the rule base list is indicated by the variable 
"ATTRBASE (string, required)." 

The "RBASELIST" section also has a subsection 
"RBLIST (required) " which stores a list of data file names 
for rule bases and flags for each rulebase . 
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Software Interface 
The following describes a specification of a 
preferred Windows operating system interface between a 
shared rule -based inferencing software engine (the "server") 
and software applications which use the engine to learn and 
evaluate rule bases for real-time control (the "clients") . 
All applications, client-based and server-based, register 
three custom message numbers for communication, and use them 
to communicate commands and results between each other. The 
message numbers used are returned by the following actions. 

AdmireControlMsg = RegisterWindowMessage { "ADMIRE/WIN Control"); 
AdmirePacJcetMsg = RegisterWindowMessage ( "ADMIRE /WIN Packet"); 
AdmireFreePtrMsg = RegisterWindowMessage ("ADMIRE/WIN FreePtr"); 

Messages are sent between client and server using 
Windows procedure "PostMessage ().» This allows the rule 
base engine and clients to function asynchronously. 
Applications should not send messages using Windows 
procedure "SendMessage (),» which, unlike "PostMessage ()," 
does not give up control in the Windows cooperative 
multitasking environment. 

When a message is sent, Windows structure "wParam" 
always contains the handle of the sending window, so the 
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receiver can easily determine where to send a reply. The 
value of Windows structure "lParam" depends on the type of 
message being sent. 

A Control Message is used to initiate or terminate a 
communication or to send other application- level control 
""^messages. Accordingly, "lParam" is set as shown in the 
following table. 



HIWORD 


LOWORD 


Meaning 


1- HELLO 


0 


Client is broadcasting a request to all 
servers to initiate communication. 




1 


Free server is responding to a client . 




2 


Busy server is responding to a client. 




3 


Client wants this server - server become 
busy. 




4 


Client does not want this server - server 
becomes free. 


2 -BYE 


0 


Client or server is requesting connection 
be terminated. 



A Packet Message is used to send packets between the 
client and server once communication has been established. 
In this case, "lParam" is a pointer to the packet data, 
which lies in global shared memory. Once a packet has been 
passed to another program via this interface, the sending 
program should not attempt to access the packet data. When 
the receiving program is done with the packet, it should 
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send a Free Pointer Message back to the sender so that the 
sender can free the associated memory. 

The Free Pointer Message is sent to the original 
sender of a packet, signifying that the original receiver is 
done with the packet and that the memory associated with the 
packet can be freed. "iparam" should point to the memory to 
be freed. 

All communications packets consist of a series of 
data structures called "chunks." Each chunk has the form 
shown in the table below. 



Addresses 


Type 


Contents 


0000-0003 


ASCII chars 


Chunk type, not a null- terminated 
string. 


0004-0007 


32-bit 
integer 


Length of chunk including the 
header . 


0008-0009 


16-bit 
integer 


Offset of start of chunk body from 
start of chunk. 


OOOA-nnnn 


Various 


Chunk body. 



All packets should begin with a header chunk " *HDR" 
and end with an end chunk " *END . " Encoding the offset of 
the chunk body as noted in the table above allows more 
fields to be added to the chunk header. 

Each packet should handle only one subject, e.g. . 
loading a series of files or learning a rule base. It is 
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preferable to send multiple small packets instead of one 
large complex packet, so that the sending of information 
does not entail large delays which can disrupt the 
multitasking environment. 
5 All applications should be able to process all chunk 

types beginning with an asterisk "*." Processing other 
chunk types is optional. If an application does not 
understand one or more chunks in a packet, it should send an 
" *UNK" chunk back to the sender of the packet as part of any 
10 reply to the packet. 

The 11 *HDR " header chunk is the first chunk in any 
packet and contains subf ields in the chunk body as indicated 
in the following table. 



15 



Addresses 


Type 


Contents 


0000-0003 


32-bit 
integer 


Packet ID number. ID numbers should 
be unique within a particular session. 


0004-0007 


32-bit 
integer 


ID of the packet responding to, or 0 
if this packet is not responding to a 
previous packet . 


0008-0009 


16-bit 
integer 


Number of chunks in this packet, 
including the " *HDR and *END chunks." 


000A-000B 


2 8-bit 
integers 


Version of the specification 
supported, in the form A.B. 
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The "*UNK" chunk lists all the chunk types in a 
previous message that were not understood by the receiver. 
The chunk body thus consists of 4n bytes, where n chunk 
types were not understood, since each chunk type is a 4 -byte 
5 string. This allows the sender to compensate for an older 
receiver which does not understand newer chunk types. 

An " * ERR " chunk indicates that a chunk was 
malformed, was missing a required field, or was otherwise 
unintelligible. The body of the " *ERR" chunk contains the 
10 fields listed in the following table. 



Addresses 


Type 


Contents 


0000-0003 


32-bit 
integer 


Address of the bad chunk in the 
referenced packet. 


0004-0007 


32-bit 
integer 


Offset of the error in the chunk. 


0008-0009 


16-bit 
integer 


Type of error according to the 
following list.. 



15 


JError Type 


Meaning 




0000 


Unexpected end of packet. 




0001 


Missing required field. 




0002 


Invalid value for field. 




7FFF 


Last globally-defined error type. 


20 


8000-FFFF 


Chunk -specific errors - possible errors are 
listed with each chunk type. 
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The " *END" chunk should be the last chunk in a 
packet and has no body. 

A "*WHN" chunk states the conditions, listed in the 
following table, under which the receiver should send back a 
response or series of responses to the sender. 



Addresses 


Type 


Contents 




0000 


8-bit 


ONERROR - When errors should be 


sent . 




integer 






0001 


8-bit 


WAITONERR-What should be done 


when an 




integer 


error is sent. 




0002 


8-bit 


ONBUSY-What should be done if 


receiver 




integer 


is busy. 





The integer " ONERROR " determines when the receiver 
should send errors generated by parsing the packet. It has 
one of the values listed below. 



Value Meaning 

0 (default) Send errors as soon as they are detected - one 

error per response packet. 

1 Send errors as soon as the entire packet has been 
parsed - all errors in one response . 

2 Send errors after the command completes - prepend 
the errors to the response to the command. 
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The "WAITONERR" integer, which has one of the values 
listed below, determines whether the receiver should wait 
for a response to any error messages before proceeding. 



| Value 


Meaning 


0 (default) 


Wait for a response from the sender before 
continuing processing of the packet. 


1 


Continue processing the packet after sending any 
errors . 


The 


■ONBUSY" integer, using one of the values below, 


indicates what the receiver should do if it is unable to 


process the 


commands in the packet immediately. 


Value 


Meaning 


0 (default) 


Queue the command for processing. 


1 


Queue the command for processing. Inform the 
sender that the command has been queued. 


2 


Queue the command for processing. Inform the 
sender when the command has been queued, and 
again when the receiver starts processing the 
command . 


3 


Do not queue the command. Inform the sender the 
command could not be processed. 
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Some commands, e.g. , "WHER" and "ABRT, " which are 
described below, are not queued but instead are processed 
ahead of other queued commands. 

A " *CMD" chunk contains the main command to be 



processed in the packet and is organized as shown in the 
table below. 



| Addresses 


Type 


Contents 


00000- 


ASCII 


Command type, not a null -terminated 


0003 




string. 


| 0 004 -nnn 


Various 


Command- specif ic fields. 



A "COMM" or comment chunk contains null -terminated 
ASCII text and can be ignored safely by all applications. 

A "PRED" chunk lists dependencies for a packet, 
i.e. , lists the packet IDs whose commands should be 
completed before the current packet can be processed. If a 
" PRED" chunk is not present, the system assumes there were 
no predecessors to the current packet . The chunk body thus 
consists of n 32-bit packet ID'S, i.e. , 4n total bytes. The 
"PRED" chunk is necessary because packets can be queued 
asynchronously. For example, a packet which requests that 
rules be learned from examples should list as a predecessor 
the packet which loads the examples. The "PRED " chunk also 
allows for parallel or distributed processing of commands. 
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A "DEFS" chunk contains default values for the rule 
engine and is organized as shown in the table below. If a 
field has a value of -1 or contains an empty ASCIIZ, i.e. , 
null- terminated, string, the present value is retained. If 
5 this chunk is sent to a server, the server's default values 
are changed to those specified in this chunk for all 
subsequent commands. Commands queued ahead of this chunk 
are not affected. 



10 



15 



Addresses 


Type 




Contents 


0000-0001 


16-bit 


integer 


Maximum rule order to be learned. 


0002-0005 


32-bit 


integer 


Maximum number of rules to be 
learned. 


0006-0009 


32-bit 


float 


Small sample k for statistics. 


000A-000D 


32-bit 


integer 


Minimum number of rules which 
should agree with each rule to be 
learned. 


00OE-0O11 


32-bit 


float 


Minimum probability that learned 
rule is correct. 


0012-0015 


32-bit 


float 


Minimum rule priority to keep when 
learning rules. 


0016-0035 


ASCIIZ 


string 


Attribute base. 


0036-0055 


ASCIIZ 


string 


Rule base. 


0056-0075 


ASCIIZ 


string 


Rule base list. 


0076-0095 


ASCIIZ 


string 


Example list. 
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The "DIRS" chunk appears as shown below and lists 
all objects of the specified type that are present in server 



memory . 



Addresses 


Type 


Loncencs 




0000 


8-bit integer 


Type of objects listed, 
all objects. 


or 0 for 


0001-0002 


16-bit integer 


Number of objects listed 




0003-0004 


16 -bit integer 


Size or eacn list entry 


m u y Lcs . 


0005-???? 


Various 


List entries. 




List 


entries have the 


format shown below. 




(Offset 


Type 


Contents 




0000-001F 


ASCIIZ string 


Name of object. 




0020 


8-bit integer 


Type of object. 




0021-0024 


32-bit integer 


Number of things, e.g., 
rules, in object. 


examples , 


1 0025-0028 


32-bit integer 


Size of object in bytes 





A packet can contain any number of command chunks, 
including none. All commands in a packet should be related 
to each other. Command chunks can contain command- specif ic 
data starting at offset 0004 within the command chunk data. 
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A "WHER" command chunk is sent from a client to 
request the status of a server. This command should always 
be processed asynchronously, regardless of how many packets 
are queued when the command is received. The server sends 
back a n HERE" chunk in response. The "WHER" chunk is 
organized as shown in the following table. 



Addresses 


Type 


Contents 


0004-0007 


32-bit integer 


Type of status information 
requested, listed in table below. 


A " 


HERE" chunk contains the fields listed in the 


following table. 




J Addresses 


Type 


Contents | 


0004-0007 


32 -bit integer 


Type of status information 
requested; list of types noted 
under "WHER" command. 


| 0008-nnn 


Various 


Specific status information. | 



The "ABRT" command, which is sent from a client to a 
server to abort a command, should always be processed 
asynchronously. The command includes the fields shown in 
the following table. 
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Addresses 


Type 




Contents 


0004- 


0007 


32-bit 


integer 


Packet ID containing command. 


0008- 


OOOB 


32-bit 


integer 


Offset of command chunk in packet, 
o it aoortmg entire paocec . 


OOOC 




8-bit 


integer 


0 - abort the rest of the packet. 

1- abort this command chunk and go 
on to the next command in the 
packet . 


000D 




8-bit 


integer 


0 - abort all successors to the 
command, reference " PRED" chunk 

1- do not abort successors to the 
command . 



A "LOAD" command loads data from a file into the 
server's memory. This should be the only way rules and 
examples are loaded from disk into the client or server - 
the client should not load rules in its own routines. 



(Addresses 


Type 


Contents 


0004 


8 -bit integer 


Type of data to load. 


0005-0024 


ASCIIZ string 


Symbolic name to give data, 32 






characters,. 


0025-0125 


ASCIIZ string 


Filename to load data from, 256 






characters . 


A " 

to a file. 


SAVE" command saves data from the server's memory 
Likewise, this should be the only way rules and 



examples are saved to disk from the client or server - the 
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client should not save rules in its own routines. The 
"SAVE" command has the fields listed below. 



| Addresses 


Type 


Contents 


0004 


8-bit integer 


Type of data to save. 


0005-0024 


ASCIIZ string ' 


Symbolic name to save from, 32 
characters . 


0025-0125 


ASCIIZ string 


Filename to save data to, 256 
characters . 



A "COPY" command, which includes the fields listed 
below, copies data from an area indicated by a symbolic name 
to another area in the server's memory. 



Addresses 


Type 


Contents 


0004 


8-bit integer 


Type of data to copy. 


0005-0024 


ASCIIZ string 


Symbolic name to copy from, 32 
characters . 


0025-0044 


ASCIIZ string 


Symbolic name to copy to, 3 2 
characters . 


A »■ 


FREE" command, which includes the fields in the 


following table, frees a memory object in the server's 


memory . 






j Addresses 


Type 


Contents 


0004 


8 -bit integer 


Type of data to free. 


j 0005-0024 


ASCIIZ string 


Name of object, 32 characters. 
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A "GETD" command, which is used to get all default 
values, has no fields and returns a "DEFS" chunk. A 
corresponding "SETD" command is not needed because the 
client is able to send instead the "DEFS" chunk with any 
necessary modifications. 

A "LIST" command, organized as shown below, lists 
all structures of the specified type and returns a "DIRS n 
chunk. The DIRS chunk tells the pieces that are currently 



in memory - 


rules , 


rulebases, 


examples, attributes, etc. If 


the type is 


set to 


zero, the 


command lists all structures. 


Addresses 


Type 




Contents 


0004 


8-bit 


integer 


Type of data to list. 



The system also provides software functions such as 
the following. 

AdmireSendPacket(HWND hwndDest, LPSTR packetcontents , integer timeout) 

The function " AdmireSendPacket asynchronously sends 
a packet and times out after the number of lOths of a second 
indicated in the "timeout" field. The timeout procedure is 
necessary to avoid leaving the client in an endless loop if 
the server is inoperative, and vice versa. 

The system also provides a handshaking procedure. 
The following describes the messages sent back and forth, 
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i.e. , handshaking, that is performed to initiate 
communications, process commands, and terminate 
communications . 

When a client wishes to initiate communication, 
i ■ e . , begin using the rule engine server, it should first 
establish a connection with the server. This is done as 
indicated below by sending a series of "HELLO, n" control 
messages back and forth, where "n" is the LOWORD, i.e. , low 
data word, of "lparam" for the HELLO message. 

1. The client sends "HELLO, 0" to all top-level windows, 
i - e . . the main operating-system interfaces of 
applications, and waits for up to 3 seconds. 

2. Each free, i.e. , unattached, server responds with 
"HELLO, 1 " and then waits for a "HELLO, 3" or 
"HELLO, 4" response from the client. If the server 
receives a subsequent "HELLO, 0" command from a 
different client, it queues that "HELLO, 0" pending 
the response from the original client. Each busy, 
i • e . , connected, server responds with "HELLO, 2." 

3. If the client receives at least one "HELLO, 1" within 
the timeout period, it sends "HELLO, 3" to the server 
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to which it intends to connect and "HELLO, 4" to all 
other free servers which responded. 

4. The server which received "HELLO, 3 " responds 
"HELLO, 2" to all subsequent "HELLO, 0" commands, 
because it is now attached to a client. Servers 
which received "HELLO, 4" return "HELLO, 1" until they 
are also attached to clients. 

5. if the client times out while waiting for a 
response, it starts up another instance of the 
server application program and goes back to step 1. 
When a client wishes to stop using a rule server, it 

should negotiate an end to the connection using the 
following process. 

1. The client sends a "BYE" control message to the 
server. 

2. The server cleans up in preparation for exit by 
releasing to the operating system the memory, fonts, 
bitmaps, and other system resources it is using and 
also by sending messages back to the client during 
this period which, e.g. , warn of unsaved files. 

3. The server sends "BYE" to. the client and breaks the 
connection. Depending on the nature of the server, 
it exits or remains loaded as a free server. 
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4. The client breaks the connection. 

The currently-used system uses a command- line 

interface. The following commands are used to produce the 

system's output. 

LEARN rbname { 

varl valuel 
var2 value2 



LHS attrl 
lhs attr2 

RHS attm 

} 

The "LEARN" command learns a new rule base from 
examples and takes a list of parameters enclosed in brackets 
{}. Variables which are specified in capitals are 
mandatory; all others are taken from defaults if they are 
not present. Variable values are listed in pairs. There 
should be at least one attribute on the left-hand side and 
only one attribute on the right-hand side. The " } " bracket 
ends the parameter list for the " LEARN" command. 

FILTER rbname filtertype value 

The "FILTER" command filters the rule base with the 
types of filters listed and described below. 
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ALWAYS attr 
NEVER attr 
ONLY attr 
PROB f 
LITTLE J f 
PRIO f 
WEIGHT f 
LOWPROB f 

The "ALWAYS" filter removes rules which do not 
contain the specified attribute on the left-hand side. 
Conversely, the "NEVER" filter removes rules which do 
contain the specified attribute on the left-hand side. The 
"ONLY" filter removes rules which have anything other than 
the specified attribute on the left-hand side. 

The remainder of the filters listed above address 
threshold levels specified separately by "f." The "PROB" 
filter removes rules with an insufficient probability of 
being correct. Likewise, the » LITTLE J, " "PRIO," and 
"WEIGHT" filters remove rules wherein the J-measure, 
priority, and weight, respectively, are too low. Finally, 
the "LOWPROB" filter removes rules with an excessive 
probability of being correct. 

The "LOWPROB" filter is used to split a rule base 
into two rule bases, one with high-probability rules and the 
other with low-probability rules. For example, the 
following steps can be performed using a set of rules "Rl." 
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1 . Copy Rl to Rhi . 

2. Copy Rl to Rio. 

3. Filter Rhi with PROB 0.5. 

4. Filter Rio with LOWPROB 0.4999999. 

The result is that rule base "Rhi" contains all of the high- 
probability rules and "Rio" contains all of the rules of 
rule base "Rl" that are not in rule base "Rhi." Moving the 
low-probability rules to a separate rule base eases analysis 
of them to determine whether they contain useful 
information. 

PRUNE rbname 

The "PRUNE" command uses subsumption pruning to 
remove unneeded rules from the rule base. 

RBLIST rblname{ 

rulebasel flagsl 
rulebase2 flags2 

} 

The "RBLIST " command creates a rule base list from 
the specified rule bases and applies the rule bases in 
proper order using the specific flags. The rule base list 
should contain at least one rule base and flags should be 
separated by vertical bars " | , " e.g. . "ALLLHS | GUESS . " 
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The allowed flags have the following meanings. Flag 
"ALLLHS, " if set, indicates that the system should have 
values for all of the LHS attributes in the rule base before 
applying the rule base. A set "GUESS" flag forces the 

3_ system to guess the most likely RHS if no rules fire. If 

the " OVERWRITE " flag is set, the system determines a new RHS 
value even if the current RHS value is known. Output data 
from each inference is kept if the "KEEPOD" flag is set. 
Finally, a set " RANDOM " flag indicates that if more than one 

10 RHS value is possible, one should be picked randomly based 
on the probabilities of the values. 
TEST name WITH exlist 

The "TEST" command tests the rule base or rule base 
list with the example set and prints the test statistics. 
15 Testing a rulebase with a set of examples involves, for each 
example in turn, comparing the expected result from the 
example with the predicted result from the rulebase. 

The "TEST" command then prints out statistics such 
as those in the illustration below. 
20 Total examples: 3134 

Examples classified: 3070 (98%) 

Examples classified correctly: 1477 (48%) 
Histogram of examples vs. rules fired per example: 
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Rules 


Exarr 


0 


64 


1 


6 


2 


53 


3 


50 


4 


108 


5 


210 


e 


252 


7 


363 


8 


454 


9 


395 


10 


302 


11 


305 


12 


239 


13 


198 


14 


61 


15 


45 


16 


25 


17 


2 


18 


2 



Average rules per example: 8.551 

Histogram of examples vs. popularity of right answer: 
Place Examples Avg. Rules 
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1 1477 8.793 

2 597 8.625 

3 235 9.311 

4 147 10.374 

5 44 10.727 

6 9 11.111 



No rules predicted correct RHS : 625 0.000 

In this illustration, the rule base was tested with 
a set of 3134 examples. If no rules fire, the rulebase does 
not make a classification. In 3070 of the examples, at 
least one rule fired. In 1477 of the examples, the rule 
base correctly classified the example. 

The next section of the analysis shows a histogram 
of the number of rules fired. The histogram peaks at 8 
rules per example and has an average of 8.551 rules per 
example . 

The last section shows details about how 
successfully the rule base chose or at least suggested the 
correct answer. In 14 77 of the examples, the rule base 
chose the correct answer. In 597 of the examples, the rule 
base selected the correct answer as the second-most-likely 
answer. In 625 of the examples, the rule base did not even 
suggest the correct answer as a possible answer. 
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The following describes commands relating to real- 
time inferencing. 

INDATA idname { (^Process for setting attributes 

from other attributes *) 

5 attrl FROM attr2 

attr2 UNKNOWN 
attr3 TO val 

^ IF attrl vail THEN attr2 from attr3 

10 The " INDATA" command creates the input data and 

should have at least one attribute-value pair. All values 
are initially set to a value of "UNKNOWN." For each 
attribute, the command gets its next value according to the 
following procedure in this example. First, the value of 

15 attribute "attrl" is copied from attribute "attr2." Next, 
attribute »attr2" is set to "UNKNOWN . " Then attribute 
"attr3" is set to the specified value "val." Finally, the 
value of attribute "attr2" is copied from the value of 
attribute "attr3" only if attribute "attrl" has the value 

20 "vail." 

The values "val" and "vail" are explicitly 
specified. For example, in a harmony "INDATA," the 
following setting is made at the start of each timestep. 

FunctionO UNKNOWN 
25 Such a setting is equivalent to the following. 
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FunctionO TO "?" 

The "TO" operator can also be used to test a rule 
base which has more flexibility than is necessary at the 
moment. For instance, if a rulebase has rules for both 
major and minor keys, the following setting can be made to 
restrict use to the rules for the major key only. 

MajMin TO Major 

To ensure that an attribute's value is updated only 
under certain conditions, a directive such as the following 
can be used. 

IF Accentl ACC THEN FunctionLA from Functionl 

This directive copies the value from the previous timestep's 
function "Functionl" into the previous accented beat's 
function "FunctionLA M only if the previous timestep was 
accented, i.e. , "Accentl" had the value "ACC." 

REALTIMEMIDI { 
rblist 

indata idname 

} 
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The " REALTIMEMIDI " command harmonizes a melody in 
real time and expects the input data to contain the 
following attributes: MelodyO, FunctionO, InversionO, 
AltoO, and TenorO . The rule base list to use, if not the 
default, is specified by "rblist." Likewise, the input data 
to use, if not the default data, is specified by applying 
the "indata." 

NEW type name n 

The "NEW " command creates a new empty structure 
capable of holding n elements, e.g. , "NEW RBLIST simpleharm 
16." Rule base lists are composed of rule bases which in 
turn are composed of rules. Likewise, example lists are 
composed of examples and attribute bases are composed of 
attributes . 

JOIN name AND name INTO name 

The "JOIN" command allows two rule bases to be 
merged to create a new rule base. 

F. Other Embodiments 

The embodiments described above are but examples, 
which can be modified in many ways within the scope of the 
appended claims. For example, the invention can also use 
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accent-based conversion, wherein additional example fields 

are allowed to be created for previous timesteps which start 

at the beginning of a beat, accented beat, or fermata. In 

accent-based conversion, only one example is created per 

timestep, so it is not necessary to weight the examples, a 

list of which would likely appear as follows. 

%NAME 0 FunctionLastAccentedBeat 
%NAME 1 FunctionLastBeat 
%NAME 2 Functionl 
%NAME 3 FunctionO 









I 




I 


I 


I 


I 


I 


I 


IV 


I 


IV 


IV 


vi 


I 


IV 


vi 


V 



With accent-based conversion, it is possible for the first 
three fields to refer to the same timestep if the previous 
timestep was at the start of an accented beat. Such 
redundancy, which leads to highly interdependent rules, 
makes real-time independence pruning essential. 

Furthermore, the invention can use non-MIDI input 
sources, such as pitch data from a microphone, allowing a 
vocalist to sing or hum a tune which is converted into 
pitches and used to generate a harmony. Likewise, the 
invention can accept pitch data from a program, such as a 
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program according to the invention which generates melodies 
instead of harmonies. 

In addition, the invention can be applied to assist 
in the derivation of a' representation for the overall 
structure of a piece of music by encoding information about 
phrases and sections in music, such as the verse-chorus 
structure common to much vocal music. The invention can 
also provide a system which includes cues for modulation 
from one key to another. 

In addition, the invention can provide a system 
allowing voices to make jumps over awkward intervals such as 
tritones or over distances further than an octave. 
Furthermore, the invention can provide a system realizing a 
figured bass that allows two voices to cross or to play in 
unison, i^e,,, play the same pitch. The invention can also 
provide a system that develops information about whether 
voices are changing pitch in the same or different direction 
as other voices. 

Moreover, the invention can provide a system that 
detects ornaments, described above, which are usually used 
to smooth a voice line by removing large jumps in pitch. 
The invention can add such ornaments to generated harmonies 
to make them more interesting. 
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Furthermore, the invention can provide a system 
relating to drums and other percussion instruments, by using 
a notation for rhythm. 

In addition, the invention can provide a system 
relating to orchestration and part writing in the areas of 
music involving expansion of four-part harmony into 
sufficient additional lines so that each instrument in an 
orchestra has something interesting to play, in the pitch 
range which the instrument can generate. The invention can 
also assist in research focusing on the methods used to 
duplicate and modify voice lines to produce distinct parts, 
and ways of moving the melody between instruments. 

Likewise, the invention can provide a system 
relating to similar concepts needed to reproduce 
contemporary music, wherein the harmonic information is 
distributed between a vocalist, lead guitar, bass guitar, 
keyboard player, and other instruments. 

In addition, the invention can use Bach inventions, 
sinfonias, and fugues to learn rules for counterpoint and 
development of a theme or motive. Similarly, the invention 
can assist in the study of methods for employing chord 
accents in syncopated rhythms to provide extracts from 
ragtime pieces by Scott Joplin, for instance. Furthermore, 
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the invention can use, for example, African drum music or 
any other sound to develop rhythm notation. 

Moreover, the invention can assist in research 
focusing on the differences between the styles of various 
composers to determine, e.g. , what makes Mozart piano 
sonatas sound different than Beethoven piano sonatas, and 
how the choral works of Bach differ from those of Handel. 

Other embodiments : 

-Extending Temporal Knowledge 
Existing rulebase sets look only at the accent of the 
current chord and the information from the previous few 
chords. This limits the ability of the rulebases to 
compensate for and generate harmonic transitions on a larger 
scale . 

Deriving a representation for the overall structure of 
a piece of music would allow ADMIRE additional flexibility 
in this regard. Such a representation would encode 
information about phrases and sections in music, such as the 
verse-chorus structure common to much vocal music. It would 
also include cues for modulation from one key to another. 
- Counterpoint and Voice Leading 
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Although the existing voice position rules perform an 
acceptable job of filling in the pitches used by a given 
chord, they do little to make the individual voices 
singable. Voices often have jumps over awkward intervals 
such as tritones or distances over an octave. Furthermore, 
the current method for realizing a figured bass does not 
allow two voices to play a unison (play the same pitch) , nor 
does it allow voices to cross. It also lacks information 
about whether voices are changing pitch in the same or 
different direction as other voices. 

Additional adding of ornamentation can be used to 
smooth a voice line by removing large jumps in pitch. Once 
ornaments are well understood, they could also be added to 
generated harmonies to make them more interesting. 

6.3 Rhythm Notation and Percussion 

Most contemporary music includes drums and other 
percussion instruments. Drum parts tend to change on a 
measure-by-measure basis, and an entire piece of music may 
contain relatively few distinct drum patterns which are 
combined in various orders. In addition, most percussion 
sounds are to a large extent atonal; the information 
contained in their parts is almost entirely rhythmic. These 
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differences will necessitate a notation for rhythm that is 
much different than the pitch-based or chord-based 
representations currently used in ADMIRE> 

Orchestration and Part Writing 
5 Orchestration and part writing are the areas of music 

involving expansion of four-part harmony into sufficient 
additional lines so that each instrument in an orchestra has 
something interesting to play, in the pitch range which the 
instrument can generate. Research here could focus on the 
10 methods used to duplicate and modify voice lines to produce 
distinct parts, and ways of moving the melody between 
instruments. 

Different Forms of Music 
Once the rules of Bach chorales are well understood, 
15 research could be expanded to encompass other musical forms. 
Bach inventions, sinfonias, and fugues could be used to 
learn rules for counterpoint and development of a theme or 
motive. Methods for employing chord accents in syncopated 
rhythms could be extracts from ragtime pieces by Scott 
20 Joplin. Rhythm notation could be developed on African drum 
music. Orchestral works by Mozart and Haydn could be used 
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as examples for part writing and orchestration, with Beatles 
music serving in a similar role for contemporary music. 

Research could also focus on the differences between 
the styles of various composers. What makes Mozart piano 
sonatas sound different than Beethoven piano sonatas, and 
how do the choral works of Bach differ from those of Handel? 
Since the algorithms used are all rule-based, it is possible 
to investigate the rules which are generated and how they 
are fired. 

All of these modifications are intended to be 
encompassed within the following claims, in which: 
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What is claimed is: 

1. A method of composing music, comprising: 

receiving a first series of musical notes defining a 
first melody having a first harmony; 

analyzing the first harmony within the first melody, 
by forming examples from the first series of musical notes, 
and deriving, in real-time, at least first and second rules 
relating to the first melody, the second rule conflicting 
with the first rule, and each of said first and second rules 
including a weight associated therewith; 

receiving additional notes of said melody and 
forming additional examples from said additional notes; 

determining ones of said additional examples that 
agree with said first rule and increasing a weight of said 
first rule when an example agrees with said first rule, and 
determining ones of said additional examples that agree with 
said second rule and increasing a weight of said second rule 
when an example agrees with said second rule; 

receiving another melody to which a harmony is to be 

formed; 
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evaluating said another melody using both of said 
first and second rules; and 

when both said first and second rules each apply to 
said another melody, applying the one of said rules which 
has the higher weight to said melody, in real-time. 

2. The method of claim 1 further comprising 
analyzing the first harmony to derive in real-time a 

second rule relating the first melody to first harmony, the 

second rule conflicting with the first rule; 

comparing the first rule to the second rule; and 
assigning a weight to an outcome of the second rule; 

and 

choosing an outcome based on the weights. 

3. A method of analyzing musical information, 
comprising : 

converting the musical information from MIDI format 
to figured bass format; 

generating an example table from the figured bass 
musical information; and 
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determining a plurality of rules, each rule 
determined from two distinct examples within said example 
table, which are different than one another, one property of 
each rule relating to statistics of musical information in 
the examples . 

4. The method of claim 3, further comprising 
applying filtering, segmentation, and subsumption 

pruning to the rule,- and 

generating dependency data using the rule. 

5. The method of claim 3, wherein deriving a rule 
using the example table comprises 

calculating a hash value for an example; 
forming a preliminary rule linking the hash value to 
an attribute to be inferenced; and 

subjecting the preliminary rule to a quality test, 

6. The method of claim 5, further comprising: 
calculating hash values for a plurality of examples,- 

and 
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wherein the subjecting comprises rejecting the 
preliminary rule if an insufficient quantity of examples 
correspond to the preliminary rule's hash value. 

5 7. The method of claim 5, further comprising: 

calculating hash values for a plurality of examples; 

and 

the quality test comprises rejecting the preliminary 
rule if the preliminary rule's hash value corresponds to an 
10 insufficient quantity of examples having a particular value 
of the attribute to be inferenced. 

8. The method of claim 5 further comprising 
calculating a J-measure for the preliminary rule, 

wherein 

15 the quality test comprises rejecting the preliminary 

rule if the preliminary rule's J-measure is insufficient. 

9. The method of claim 4 wherein the rule is 
filtered out if the rule disregards a current melody note in 
determining a chord function. 



20 



10. The method of claim 4 further comprising 
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deriving a plurality of rules; 
organizing the rules in a rulebase; and 
segmenting the rulebase into a plurality of new 
rulebases ; wherein 

a first new rulebase includes rules having a 
desired attribute; and 

a second new rulebase includes rules lacking 
the desired attribute. 

11. A method of producing a database of rules for 
producing musical sounds, comprising: 

using first musical sounds as examples to derive a 
plurality of rules; 

organizing the rules in a rulebase; and 
removing a first rule from the rulebase if: 

the first rule and a second rule predict a same 
value of a same attribute, 

the first rule has more attributes than the 

second rule, 

all of the attributes of the first rule are 
present with substantially the same values in the second 
rule , and 
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the second rule is correct at least as often as 
the first rule. 

12. The method of claim 11 further comprising 
determining that two rules are dependent if both 

rules are activated in at least half of the instances in 
which at least one of the two rules is activated. 

13. A music composition system comprising 

an analyzer receiving a first harmony including a 
first melody and deriving in real-time a first rule relating 
the first melody to the first harmony; and 

a harmonizer receiving a second melody and applying 
the first rule in real-time to the second melody to produce 
a second harmony relating to the second melody. 

14 . A music composition system comprising 

an analyzer receiving a first harmony including a 
first melody and deriving in real-time a first rule relating 
the first melody to the first harmony and a weight for the 
first rule based on statistical information in the first 
melody and first harmony, wherein the analyzer derives a 
second rule in real-time relating the first melody to first 
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harmony and a weight for the first rule based on statistical 
information, the second rule conflicting with the first 
rule; and 

a harmonizer receiving a second melody and applying 
the first rule in real-time to the second melody to produce 
a second harmony relating to the second melody, 

said harmonizer comparing the first rule to the 
second rule and determining which of said rules to use based 
on said weights. 

15. A method of converting musical information of a 
musical piece from MIDI format to figured bass format, 
comprising 

transposing the musical piece to a standard key; 

segmenting the transposed musical piece into chords 
by beginning a new chord whenever a voice changes pitch; 

attempting to match each chord with a known chord to 
produce identified chords each having a root and a type; 

determining a position for each voice of each 
identified chord by comparing each voice's pitch with 
pitches allowed in the voice's matching known chord; and 
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attempting to match each identified chord with a 
known function by comparing each identified chord's root and 
type with a table of common functions. 

16. A musical information analyzer, comprising 

a converter receiving musical information in MIDI 
format and producing musical information in figured bass 
format ; 

a table generator deriving an example table from the 
figured bass musical information; and 

a rule generator, determining a plurality of rules, 
each rule determined from the two distinct examples which 
are different than one another, one property of each rule 
relating to statistics of musical information in the 
examples . 

17. The analyzer of claim 16, further comprising 
a filter applying filtering to the rule; 

a rule segmenter applying segmentation to the rule; 
a pruner applying subsumption pruning to the rule; 

and 

a dependence analyzer generating dependence data 
using the rule. 
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18. The analyzer of claim 16 , wherein the rule 
generator comprises : 

a hash calculator calculating a hash value for an 

example; 

a preliminary rule generator forming a preliminary 
rule linking the hash value to an attribute to be 
inferenced; and 

a tester subjecting the preliminary rule to a 
quality test. 

19. The analyzer of claim 18, wherein 

the hash calculator calculates hash values for a 
plurality of examples; and 

the quality test comprises rejecting the preliminary 
rule if an insufficient quantity of examples correspond to 
the preliminary rule's hash value. 

20. The analyzer of claim 18, wherein 

the hash calculator calculates hash values for a 
plurality of examples; and 

the quality test comprises rejecting the preliminary 
rule if the preliminary rule's hash value corresponds to an 
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insufficient quantity of examples having a particular value 
of the attribute to be inferenced. 

21. The analyzer of claim 18 further comprising 

a J-measure calculator calculating a J-measure for 
the preliminary rule, wherein 

the quality test comprises rejecting the preliminary 
rule if the preliminary rule's J-measure is insufficient. 

22. The analyzer of claim 17 wherein the filter 
removes the rule if the rule disregards a current melody 
note in determining a chord function. 

23. The analyzer of claim 17 wherein 

the rule generator derives a plurality of rules; 

a rule organizer organizes the rules in a rule base; 

and 

the rule segmenter segments the rule base into a 
plurality of new rule bases; wherein 

a first new rule base contains rules having a 
desired attribute; and 

a second new rule base contains rules lacking the 

desired attribute. 
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24. The analyzer of claim 17 wherein: 

the rule generator derives a plurality of rules; 

a rule organizer organizes the rules in a rulebase; 

and 

the pruner removes a first rule from the rulebase 

if: 

the first rule and a second rule predict a same 
value of a same attribute, 

the second rule has more attributes than the 

first rule, 

all of the attributes of the first rule are 
present with the same values in the second rule, and 

the second rule is correct at least as often as 
the first rule. 

25. The analyzer of claim 17 wherein 

the rule generator derives a plurality of rules; and 
the dependence analyzer determines that two rules 
are dependent if said two rules are activated in at least 
half of the instances in which at least one of the two rules 
is activated. 
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Kindly add the following new claims : 

26. A system which converts musical information of 
a musical piece from MIDI format to figured bass format, 
, comprising 

5 a key transposer transposing the musical piece to a 

standard key; 

a segmenter segmenting the transposed musical piece 
into chords by beginning a new chord whenever a voice 
changes pitch; 

10 a chord matcher attempting to match each chord with 

a known chord to produce identified chords each having a 
root and a type; 

a position determiner determining a position for 
each voice of each identified chord by comparing each 
15 voice's pitch with pitches allowed in the voice's matching 
known chord; and 

a function matcher attempting to match each 
identified chord with a known function by comparing each 
identified chord's root and type with a table of common 
functions . 
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27. A method of composing music, comprising: 
obtaining a sample of music whose style is to be 
analyzed; 

producing a plurality of examples from said sample 
of music; 

generating a plurality of rules from the plurality 
of examples, said rules predicting certain examples which 
follow other examples, and each said rule including weights 
associated therewith, said weights defining a statistical 
likelihood that said rule will be followed, 

increasing a weight of a rule when a particular 
example agrees with the rule; and 

decreasing a weight of the rule when a particular 
example does not agree with the rule. 

28. A method as in claim 27 further comprising: 

storing all of said rules into a rulebase; 

obtaining a melody which is to be analyzed using 
said rules in said rulebase; and 

analyzing said melody using all of said rules in 
said rulebase, by using said melody to fire all rules in 
said rulebase which are applicable to said melody, 
evaluating a result of firing of said rules, and resolving 
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conflicts between conflicting rules based on said weights 
associated with the conflicting rules. 



29. A method as in claim 28 wherein said rules 
relate to harmonies that are derived from melodies, and 
further comprising: 

presenting a harmony produced by a particular rule 
to an operator who can determine if said harmony is 
desirable ; 

accepting an input from said operator indicating if 
said harmony is desirable; 

increasing the weight for the particular rule if the 
harmony is desirable and decreasing the weight for the 
particular rule if the policy is not desirable. 

30. A method of generating rules from a musical 
piece, comprising: 

obtaining musical information; 

converting said musical information to examples; 

determining a minimum number parameter, indicating a 
minimum number of agreements before a rule can be formed; 

comparing said examples to generate a prediction of 
attributes that will follow one another; 

determining if each said prediction has occurred 
before within said set of examples by a number of times 
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having a predetermined relationship with said minimum number 
parameter; 

establishing a rule of the form "If (a) Then ( b ) " if 
said prediction has occurred said number of times having 
said predetermined relationship with said minimum number 
parameter; and 

establishing a weight associated with said rule, 
said weight indicative of a number of times that (a) 
correctly predicts (b) . 

31. A method as in claim 30 wherein said rule is of 
the form "If attribute (Al) and attribute (A2) Then 
attribute (B3); correct X percent of the time, where x is 
the percentage of times that attributes (Al) and (A2) 
predict attribute (B3) . 

32. A method as in claim 27, further comprising 
ordering said database in a way that improves use of said 
rules. 

33. A method as in claim 32, wherein said ordering 
comprises 

determining a certain attribute which is important 
for a current application; and 
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filtering the plurality of rules to prevent rules 
from being used which do not use that attribute. 

34. A method as in claim 33 wherein said attribute 
is a rule which disregards a current melody note in 
determining a current chord function. 

35. A method as in claim 32, wherein said ordering 
comprises 

determining a desired attribute for a desired 
application; 

grouping the plurality of rules based on whether 
they include that desired attribute; 

placing rules which include the desired attribute in 
a first segmented rulebase, and placing rules which do not 
include the desired attribute into a second unsegmented 
rulebase . 

36. A method as in claim 35 further comprising: 
obtaining a musical melody to be applied to said 

database; 

first checking said segmented rulebase to determine 
if rules in said segmented rulebase meet a predetermined 
criteria and if so, using only the rules in said segmented 
rulebase; and 
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if no rules meet the predetermined criteria, using 
the rules in said unsegmented rulebase . 



37. A method as in claim 36 wherein the 
predetermined criteria is whether a rule has fired. 

38. A method as in claim 27, further comprising 
analyzing the rules to determine rules which are depending 
with other rules; and 

removing at least some of the dependent rules. 

39. A method as in claim 38 wherein said analyzing 
comprises : 

finding at least two rules which produce a same 

result; 

determining a set of examples for which each rule 

fires ; 

determining an overlap for which both rules fire; 

and 

determining a percentage of dependence between the 

rules. 

40. A method of composing music, comprising: 
obtaining a sample of music whose style is to be 
analyzed; 
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producing a plurality of examples from said sample 
of music; 

generating a plurality of rules from the plurality 
of examples, said rules predicting certain examples which 
follow other examples, and each said rule including weights 
associated therewith, said weights defining a statistical 
likelihood that said rule will be followed; 

storing all of said rules into a rulebase; 

analyzing a melody which using said rules in said 
rulebase to form a harmony accompanying said melody to 
provide an accompaniment to said melody according to said 
rulebase ; 

listening to said accompaniment; and 

either taking no action based on said accompaniment 
in which case a weight which produced the harmony is 
unchanged, taking an action to indicate dislike of the 
result in which case said weight which produced the harmony 
is decreased, or taking an action to indicate like of the 
result in which case said weight is increased. 

41. A method as in claim 40 wherein said increase 
in weight is by .01. 
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42. A method as in claim 1, wherein said first and 
second rules are increased in weight each time an example 
agrees . 



43. A method as in claim 1, wherein there are more 
than two rules formed by said analyzing, said more than two 
rules form a rulebase, and wherein all of said rules in said 
rulebase are evaluated during said evaluating, 

44. A system as in claim 14, wherein there are more 
than two rules formed by said analyzer, said more than two 
rules form a rulebase, and wherein all of said rules in said 
rulebase are evaluated by said analyzer. 

45. A method of composing music, comprising: 
obtaining a sample of music whose style is to be 

analyzed; 

producing a plurality of examples from said sample 
of music; 

generating a plurality of rules from the plurality 
of examples, said rules predicting certain examples which 
follow other examples, and each said rule including weights 
associated therewith, said weights defining a statistical 
likelihood that said rule will be followed; 

storing all of said rules into a rulebase; 
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using said rulebase to analyze another melody, by 
evaluating taking all of the plurality of rules in the 
rulebase in parallel and then resolves any conflicts between 
rules based on the rule weights. 1. A method of composing 
music, comprising 

receiving a first series of musical notes defining a 
first melody; 

analyzing the first harmony to derive in real-time 
at least two rules relating to the first melody; 

receiving additional notes; and 

applying the at least two rules in real-time to the 
additional notes, to generate chord notes that relate to the 
additional notes. 
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